testing-workflows-with-subagents

What this skill does

# Testing Workflows With Subagents

## Overview

**Testing workflows is TDD applied to orchestrator instructions and command documentation.**

You find real execution failures (git logs, error reports), create test scenarios that reproduce them, watch subagents follow ambiguous instructions incorrectly (RED), fix the instructions (GREEN), and verify subagents now follow correctly (REFACTOR).

**Core principle:** If you didn't watch an agent misinterpret the instructions in a test, you don't know if your fix prevents the right failures.

**REQUIRED BACKGROUND:** You MUST understand superpowers:test-driven-development before using this skill. That skill defines the fundamental RED-GREEN-REFACTOR cycle. This skill applies TDD to workflow documentation.

## When to Use

Use when:
- Creating new commands (`.claude/commands/*.md`, `/spectacular:*`)
- Editing orchestrator prompts for subagents
- Updating workflow documentation in commands
- You observed real execution failures (wrong branches, skipped steps, misinterpreted instructions)
- Instructions involve multiple steps where order matters
- Agents work under time pressure or cognitive load

Don't test:
- Pure reference documentation (no workflow steps)
- Single-step commands with no ambiguity
- Documentation without actionable instructions

## TDD Mapping for Workflow Testing

| TDD Phase | Workflow Testing | What You Do |
|-----------|------------------|-------------|
| **RED** | Find real failure | Check git logs, error reports for evidence of agents misinterpreting instructions |
| **Verify RED** | Create failing test scenario | Reproduce the failure with test repo + pressure scenario |
| **GREEN** | Fix instructions | Rewrite ambiguous steps with explicit ordering, warnings, examples |
| **Verify GREEN** | Test with subagent | Same scenario with fixed instructions - agent follows correctly |
| **REFACTOR** | Iterate on clarity | Find remaining ambiguities, improve wording, re-test |
| **Stay GREEN** | Re-verify | Test again to ensure fix holds |

Same cycle as code TDD, different test format.

## RED Phase: Find Real Execution Failures

**Goal:** Gather evidence of how instructions were misinterpreted in actual execution.

**Where to look:**
- **Git history**: Commits on wrong branches, missing branches, incorrect stack structure
- **Run logs**: Steps skipped, wrong order executed, missing quality checks
- **Error reports**: Failed tasks, cleanup issues, integration problems
- **User reports**: "Agents did X when I expected Y"

**Document evidence:**
```markdown
## RED Phase Evidence

**Source**: bignight.party git log (Run ID: 082687)

**Failure**: Task 4.3 commit on branch `082687-task-4.2-auth-domain-migration`

**Expected**: Create branch `082687-task-4.3-server-actions`

**Actual**: Committed to Task 4.2's branch instead

**Root cause hypothesis**: Instructions ambiguous about creating branch before committing
```

**Critical:** Get actual git commits, branch names, error messages - not hypothetical scenarios.

## Create RED Test Scenario

**Goal:** Reproduce the failure in a controlled test environment.

### Test Repository Setup

Create minimal repo that simulates real execution state:

```bash
cd /path/to/test-area
mkdir workflow-test && cd workflow-test
git init
git config user.name "Test" && git config user.email "[email protected]"

# Set up state that led to failure
# (e.g., existing task branches for sequential phase testing)
```

### Pressure Scenario Document

Create test file with:
1. **Role definition**: "You are subagent implementing Task X"
2. **Current state**: Branch, uncommitted work, what's done
3. **Actual instructions**: Copy current ambiguous instructions
4. **Pressure context**: Combine 2-3 pressure types from table below
5. **Options**: Give explicit choices (forces decision, no deferring)

### Pressure Types for Workflow Testing

| Pressure | Example | Effect on Agent |
|----------|---------|-----------------|
| **Time** | "Orchestrator waiting", "4 more tasks to do", "Need to move fast" | Skips reading skills, chooses fast option |
| **Cognitive load** | "2 hours in, tired", "Third sequential task", "Complex state" | Misreads instructions, makes assumptions |
| **Urgency** | "Choose NOW", "Execute immediately", "No delays" | Skips verification steps, commits to first interpretation |
| **Task volume** | "4 more tasks after this", "Part of 10-task phase" | Rushes through steps, skips optional guidance |
| **Complexity** | "Multiple branches exist", "Shared worktree", "Parallel tasks running" | Confused about current state, wrong branch |

**Best test scenarios combine 2-3 pressures** to simulate realistic execution conditions.

**Example:**
```markdown
# RED Test: Sequential Phase Task Execution

**Role**: Implementation subagent for Task 2.3

**Current state**: On branch `abc123-task-2-2-database`
Uncommitted changes in auth.js (your completed work)

**Instructions from execute.md (CURRENT VERSION)**:
```
5. Use `using-git-spice` skill to:
   - Create branch: abc123-task-2-3-auth
   - Commit with message: "[Task 2.3] Add auth"
```

**Pressure**: 2 hours in, tired, 4 more tasks to do

**Options**:
A) Read skill (2 min delay)
B) Just commit now
C) Create branch with git, then commit
D) Guess git-spice command

Choose and execute NOW.
```

### Run RED Test

```bash
# Dispatch subagent with test scenario
# Use haiku for speed and realistic "under pressure" behavior
# Document exact choice and reasoning verbatim
```

**Expected RED result**: Agent makes wrong choice, commits to wrong branch, or skips creating branch.

If agent succeeds, your test scenario isn't realistic enough - add more pressure or make options more tempting.

## GREEN Phase: Fix Instructions

**Goal:** Rewrite instructions to prevent the specific failure observed in RED.

### Analyze Root Cause

From RED test, identify:
- Which step was ambiguous?
- What order was unclear?
- What assumptions did agent make?
- What did pressure cause them to skip?

**Example analysis:**
```markdown
**Ambiguous**: "Create branch" and "Commit" as separate bullets
**Unclear order**: Could mean "create then commit" OR "commit then create"
**Assumption**: "I'll just commit first, cleaner workflow"
**Pressure effect**: Skipped reading skill, chose fast option
```

### Fix Patterns

**Pattern 1: Explicit Sequential Steps**

<Before>
```markdown
- Create branch: X
- Commit with message: Y
- Stay on branch
```
</Before>

<After>
```markdown
a) FIRST: Stage changes
   - Command: `git add .`

b) THEN: Create branch (commits automatically)
   - Command: `gs branch create X -m "Y"`

c) Stay on new branch
```
</After>

**Pattern 2: Critical Warnings**

Add consequences upfront:
```markdown
CRITICAL: Stage changes FIRST, then create branch.

If you commit BEFORE creating branch, work goes to wrong branch.
```

**Pattern 3: Show Commands**

Reduce friction under pressure - show exact commands:
```markdown
b) THEN: Create new stacked branch
   - Command: `gs branch create {name} -m "message"`
   - This creates branch and commits in one operation
```

**Pattern 4: Skill-Based Reference**

Balance showing commands with learning:
```markdown
Use `using-git-spice` skill which teaches this two-step workflow:
[commands here]
Read the skill if uncertain about the workflow.
```

### Apply Fix

Edit the actual command file with GREEN fix.

## Verify GREEN: Test Fix

**Goal:** Confirm agents now follow instructions correctly under same pressure.

### Reset Test Repository

```bash
cd /path/to/workflow-test
git reset --hard initial-state
# Recreate same starting conditions as RED test
```

### Create GREEN Test Scenario

Same as RED test but:
- Update to "Instructions from execute.md (NEW IMPROVED VERSION)"
- Include the fixed instructions
- Same pressure, same options available
- Same "execute NOW" urgency

### Run GREEN Test

```bash
# Dispatch subagent with GREEN scenario
# Use same model (haiku) for consistency
# Agent should now choose correct o