ds-delegate

What this skill does

## Contents

- [The Iron Law of Delegation](#the-iron-law-of-delegation)
- [Core Principle](#core-principle)
- [The Process](#the-process)
- [Delegation Facts](#delegation-facts)

<EXTREMELY-IMPORTANT>
## The Iron Law of Delegation

**YOU MUST route EVERY ANALYSIS STEP THROUGH A TASK AGENT. This is not negotiable.**

You MUST NOT:
- Write analysis code directly
- Run "quick" data checks
- Edit notebooks or scripts
- Make "just this one plot"

**If you're about to write analysis code in main chat, STOP. Spawn a Task agent instead.**
</EXTREMELY-IMPORTANT>

## Core Principle

**Fresh subagent per task + output-first verification = reliable analysis**

- Analyst subagent does the work
- Must produce visible output at each step
- Methodology reviewer checks approach
- Loop until output verified

## When to Use

Called by `ds-implement` for each task in PLAN.md. Don't invoke directly.

## The Process

```
For each task:
    1. Dispatch analyst subagent
       - If questions → answer, re-dispatch
       - Implements with output-first protocol
    2. Verify outputs are present and reasonable
    3. Dispatch methodology reviewer (if complex)
    4. Mark task complete, log to LEARNINGS.md
```

## Task Type Detection

Each task in PLAN.md should have a `type` field. Detect and route accordingly:

| Task Type | Agent | Constraints | Example Tasks |
|-----------|-------|-------------|---------------|
| `engineering` | `workflows:ds-engineer` | ds-engineering-constraints.md index + atomic E1-E5 files | ETL, merge, clean, transform, pipeline, schema, join |
| `analysis` | `workflows:ds-analyst` | ds-analysis-constraints.md index + atomic A1-A7 files | regression, test, model, visualize, estimate, summarize |

**Detection heuristic (when type field is missing):**

| Task contains these keywords | Type |
|------------------------------|------|
| merge, join, clean, ETL, transform, pipeline, ingest, schema, deduplicate, normalize | engineering |
| regression, estimate, test, model, plot, chart, visualize, summarize, correlate, panel | analysis |
| ambiguous | Default to `analysis` (safer — analysis constraints are stricter) |

## Step 1: Dispatch Analyst/Engineer

**Pattern:** Use structured delegation template from `references/delegation-template.md`

Every delegation MUST include:
1. TASK - What to analyze
2. EXPECTED OUTCOME - Success criteria
3. REQUIRED SKILLS - Statistical/ML methods needed
4. REQUIRED TOOLS - Data access and analysis tools
5. MUST DO - Output-first verification
6. MUST NOT DO - Methodology violations
7. CONTEXT - Data sources and previous work
8. VERIFICATION - Output requirements

Use this Task invocation (fill in brackets). **Route based on task type detected above:**

*All paths below are relative to this skill's base directory.*

**For `analysis` tasks:**
```
Task(subagent_type="workflows:ds-analyst", prompt="""
# TASK

Analyze: [TASK NAME]

## EXPECTED OUTCOME

You will have successfully completed this task when:
- [ ] [Specific analysis output 1]
- [ ] [Specific analysis output 2]
- [ ] Output-first verification at each step
- [ ] Results documented with evidence

## REQUIRED SKILLS

This task requires:
- [Statistical method]: [Why needed]
- [Programming language]: Data manipulation
- Output-first verification (mandatory)
- SQL reference: Read `${CLAUDE_SKILL_DIR}/../../skills/ds-delegate/references/sql-patterns.md` for dialect-specific patterns
- Data quality checks: Read `${CLAUDE_SKILL_DIR}/../../skills/ds-implement/references/ds-checks.md` for DQ1-DQ6 verification patterns (mandatory)
- Analysis constraints: Read `${CLAUDE_SKILL_DIR}/../../references/constraints/ds-analysis-constraints.md` for the constraint index, then load:
  Read `${CLAUDE_SKILL_DIR}/../../references/constraints/ds-robustness-checks.md`
  Read `${CLAUDE_SKILL_DIR}/../../references/constraints/ds-standard-error-spec.md`
  Read `${CLAUDE_SKILL_DIR}/../../references/constraints/ds-visualization-integrity.md`
  Read `${CLAUDE_SKILL_DIR}/../../references/constraints/ds-table-figure-pairing.md`
- Analysis conventions: Read `${CLAUDE_SKILL_DIR}/../../references/constraints/ds-common-conventions.md` for the convention index, then load:
  Read `${CLAUDE_SKILL_DIR}/../../references/constraints/ds-statistical-validity.md`
  Read `${CLAUDE_SKILL_DIR}/../../references/constraints/ds-p-hacking-prevention.md`
  Read `${CLAUDE_SKILL_DIR}/../../references/constraints/ds-sample-selection.md`
  Read `${CLAUDE_SKILL_DIR}/../../references/constraints/ds-deviation-rules-analysis.md`

## REQUIRED TOOLS

You will need:
- Read: Load datasets and existing code
- Write: Create analysis scripts/notebooks
- Bash: Run analysis and verify outputs

**Tools denied:** None (full analysis access)

## MUST DO

- [ ] Print state BEFORE each operation (shape, head)
- [ ] Print state AFTER each operation (nulls, sample)
- [ ] Verify outputs are reasonable at each step
- [ ] Document methodology decisions

## MUST NOT DO

- ❌ Skip verification outputs
- ❌ Proceed with questionable data without flagging
- ❌ Guess on methodology (ask if unclear)
- ❌ Claim completion without visible outputs

## CONTEXT

### Task Description
[PASTE FULL TASK TEXT FROM PLAN.md]

### Analysis Context
- Analysis objective: [from SPEC.md]
- Data sources: [list with paths]
- Previous steps: [summary from LEARNINGS.md]

## Output-First Protocol (MANDATORY)
For EVERY operation:
1. Print state BEFORE (shape, head)
2. Execute operation
3. Print state AFTER (shape, nulls, sample)
4. Verify output is reasonable

Example:
```python
print(f"Before: {df.shape}")
df = df.merge(other, on='key')
print(f"After: {df.shape}")
print(f"Nulls introduced: {df.isnull().sum().sum()}")
df.head()
```

## Required Outputs by Operation
| Operation | Required Output |
|-----------|-----------------|
| Load data | shape, dtypes, head() |
| Filter | shape before/after, % removed |
| Merge/Join | shape, null check, sample |
| Groupby | result shape, sample groups |
| Model fit | metrics, convergence |

## If Unclear
Ask questions BEFORE implementing. Don't guess on methodology.

## Output
Report: what you did, key outputs observed, any data quality issues found.
""")
```

**For `engineering` tasks:**
```
Task(subagent_type="workflows:ds-engineer", prompt="""
# TASK

Engineer: [TASK NAME]

## EXPECTED OUTCOME

You will have successfully completed this task when:
- [ ] [Specific engineering output 1]
- [ ] [Specific engineering output 2]
- [ ] Output-first verification at each step
- [ ] Results documented with evidence

## REQUIRED SKILLS

This task requires:
- [Engineering method]: [Why needed]
- [Programming language]: Data manipulation
- Output-first verification (mandatory)
- SQL reference: Read `${CLAUDE_SKILL_DIR}/../../skills/ds-delegate/references/sql-patterns.md` for dialect-specific patterns
- Data quality checks: Read `${CLAUDE_SKILL_DIR}/../../skills/ds-implement/references/ds-checks.md` for DQ1-DQ6 verification patterns (mandatory)
- Engineering constraints: Read `${CLAUDE_SKILL_DIR}/../../references/constraints/ds-engineering-constraints.md` for the constraint index, then load:
  Read `${CLAUDE_SKILL_DIR}/../../references/constraints/ds-determinism.md`
  Read `${CLAUDE_SKILL_DIR}/../../references/constraints/ds-schema-contracts.md`
  Read `${CLAUDE_SKILL_DIR}/../../references/constraints/ds-join-audits.md`
  Read `${CLAUDE_SKILL_DIR}/../../references/constraints/ds-idempotency.md`
  Read `${CLAUDE_SKILL_DIR}/../../references/constraints/ds-error-handling.md`

## REQUIRED TOOLS

You will need:
- Read: Load datasets and existing code
- Write: Create ETL scripts/pipelines
- Bash: Run transformations and verify outputs

**Tools denied:** None (full engineering access)

## MUST DO

- [ ] Print state BEFORE each operation (shape, head)
- [ ] Print state AFTER each operation (nulls, sample)
- [ ] Verify schema contracts at each step
- [ ] Validate determinism (same input → same output)
- [ ] Check join key uniqueness before merging
- [ ] Document