Overview

Apply output-first verification at every step of analysis implementation. This is Phase 3 of the /ds workflow.

Contents

• The Iron Law of DS Implementation - EVERY step MUST produce visible output
• Delegation - Main chat orchestrates, subagents analyze
• What Output-First Means
• Red Flags
• SAS Language Routing - Load SAS enforcement when PLAN.md specifies SAS
• Implementation Process
• Verification Patterns - See references/verification-patterns.md
• Common Failures
• Gate: Exit Implementation

Implementation (Output-First Verification)

Implement analysis with mandatory visible output at every step. NO TDD - instead, every code step MUST produce and verify output.

EVERY CODE STEP MUST PRODUCE VISIBLE OUTPUT. This is not negotiable.

Before moving to the next step, you MUST:

1. Run the code
2. See the output (print, display, plot)
3. Verify output is correct/reasonable
4. Document in .planning/LEARNINGS.md
5. Only THEN proceed to next step

This applies even when YOU think:

• "I know this works"
• "It's just a simple transformation"
• "I'll check results at the end"
• "The code is straightforward"

If you're about to write code without outputting results, STOP. </EXTREMELY-IMPORTANT>

Delegation

You orchestrate. Subagents analyze. For every task in PLAN.md, use the delegation skill:

Read ${CLAUDE_SKILL_DIR}/../../skills/ds-delegate/SKILL.md and follow its instructions.

This is MANDATORY. ds-delegate contains the Task agent templates, output-first protocol details, methodology review patterns, and rationalization prevention. Do not attempt to summarize or shortcut it.

If you're about to write analysis code directly, STOP and read ds-delegate.

If you wrote analysis code in main chat, DELETE it immediately and dispatch a Task agent instead. Code written in main chat is contaminated by orchestrator context and must not be kept.

Delete & Restart Protocol

Helpfulness Check: If you kept main-chat code "because it worked," you bypassed the orchestration protocol. Working code written in the wrong place skips verification and review — it is anti-helpful to the user. Delete it. </EXTREMELY-IMPORTANT>

What Output-First Means

The Mantra: If not visible, it cannot be trusted.

Rationalization Table

Drive-Aligned Framing

Red Flags - STOP Immediately

Implementation Strategy Choice

After prerequisites pass and PLAN.md verified, check for parallelization potential:

Skip this choice when:

• PLAN.md has fewer than 4 tasks
• All tasks are dependent (every task is after N with no independent groups)
• Tasks form a pipeline (clean → merge → aggregate → model)
• CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS is not available

Otherwise, ask the user:

AskUserQuestion(questions=[{
  "question": "How should we implement the analysis tasks in PLAN.md?",
  "header": "Strategy",
  "options": [
    {"label": "Sequential (Default)", "description": "One task at a time with output-first verification. Safest, most DS work is sequential."},
    {"label": "Agent team (parallel)", "description": "Spawn analyst per independent task group. Only for truly independent analysis branches (descriptive stats by subgroup, model comparisons). Requires reconciliation."}
  ],
  "multiSelect": false
}])

If Sequential: Proceed to Implementation Process below (current behavior).

If Agent team: Skip to Agent Team Implementation (Parallel).

SAS Language Routing

If PLAN.md specifies Implementation Language: SAS or Mixed, load SAS enforcement BEFORE dispatching any SAS tasks. Paste the enforcement block into every SAS subagent prompt.

Full SAS enforcement rules: See references/sas-enforcement.md

Implementation Process Flowchart

┌─────────────────────────┐
│  Read PLAN.md + Load    │
│  ds-delegate + ETL refs │
└───────────┬─────────────┘
            ▼
┌─────────────────────────┐
│  For each task in PLAN  │◄──────────────────────┐
│  (in dependency order)  │                       │
└───────────┬─────────────┘                       │
            ▼                                     │
┌─────────────────────────┐                       │
│  Dispatch Task agent    │                       │
│  (per ds-delegate)      │                       │
└───────────┬─────────────┘                       │
            ▼                                     │
┌─────────────────────────┐     ┌──────────────┐  │
│  Read agent output      │────→│ Output wrong │  │
│  Verify output present  │     │ or missing?  │  │
│  + reasonable            │     └──────┬───────┘  │
└───────────┬─────────────┘            │           │
            │ OK                       ▼           │
            │                 ┌──────────────────┐ │
            │                 │ STOP. Investigate │ │
            │                 │ Log issue. Fix.   │ │
            │                 │ Re-verify.        │ │
            │                 └──────────────────┘ │
            ▼                                     │
┌─────────────────────────┐                       │
│  Log to LEARNINGS.md    │                       │
│  (Task N: COMPLETE)     │                       │
└───────────┬─────────────┘                       │
            ▼                                     │
        More tasks? ──── YES ─────────────────────┘
            │
            NO
            ▼
┌─────────────────────────┐
│  Exit Gate: Compare     │
│  PLAN.md vs LEARNINGS   │
│  (all tasks accounted?) │
└───────────┬─────────────┘
            ▼
┌─────────────────────────┐
│  Invoke ds-validate     │
└─────────────────────────┘

This flowchart IS the specification. If the narrative below and this flowchart disagree, the flowchart wins.

Topic Change Protocol

If user sends an off-topic message during implementation, follow C6 from ds-common-constraints.md:

1. Announce: "Pausing ds-implement to address your request."
2. Handle: Process the request (normal tools allowed outside the loop).
3. Announce: "Resuming ds-implement. Reading state files for current progress."
4. Reload: Read LEARNINGS.md and PLAN.md to restore context.
5. Resume: Continue from where you left off.

Do NOT silently switch context. Silent switches kill the implementation loop.

Implementation Process

Step 1: Read Plan, Load Shared Enforcement, and Delegation Skill

Read ${CLAUDE_SKILL_DIR}/../../references/constraints/ds-common-constraints.md for the full constraint index.

For implementation phase, load these specific constraints: Read ${CLAUDE_SKILL_DIR}/../../references/constraints/ds-data-quality-checks.md Read ${CLAUDE_SKILL_DIR}/../../references/constraints/ds-post-subagent-boundary.md Read ${CLAUDE_SKILL_DIR}/../../references/constraints/ds-deviation-rules.md

Load conventions for implementation phase: Read ${CLAUDE_SKILL_DIR}/../../references/constraints/ds-common-conventions.md for the full convention index. Read ${CLAUDE_SKILL_DIR}/../../references/conventions/ds-assumption-over-evidence.md Read ${CLAUDE_SKILL_DIR}/../../references/conventions/ds-deferred-verification.md Read ${CLAUDE_SKILL_DIR}/../../references/conventions/ds-topic-change-protocol.md Read ${CLAUDE_SKILL_DIR}/../../references/conventions/ds-escape-patterns.md

Read(".planning/PLAN.md")

Read ${CLAUDE_SKILL_DIR}/../../skills/ds-delegate/SKILL.md and follow its instructions.

Follow the task order defined in the plan. Use ds-delegate's templates for every task.

ETL Strategy Enforcement — load domain-specific references based on PLAN.md:

If PLAN.md contains an ## ETL Strategy section, the user made decisions during planning that MUST be enforced during implementation. Check each subsection and load the corresponding enforcement:

To load these references, discover the plugin cache path first:

• Read ${CLAUDE_SKILL_DIR}/../../skills/wrds/references/sas-etl.md and follow its instructions.
• Read ${CLAUDE_SKILL_DIR}/../../skills/ds-implement/references/etl-enforcement.md and follow its instructions.

If PLAN.md has NO ETL Strategy section: Skip this — proceed directly to Step 2.

Context Monitoring

Before starting each task, check context availability:

At Warning level: After current task completes, invoke: Read ${CLAUDE_SKILL_DIR}/../../skills/ds-handoff/SKILL.md and follow its instructions.

Why: A multi-task analysis pipeline with 20% context remaining produces degraded output. Better to handoff cleanly and resume fresh.

Step 2: Execute Each Task via Delegation

For each task in PLAN.md:

1. Dispatch analyst subagent (per ds-delegate pattern)
2. Verify outputs are present and reasonable
3. Dispatch methodology reviewer (for statistical tasks)
4. Log findings to LEARNINGS.md

Step 3: Log to LEARNINGS.md

Document every significant step:

## Task N: [Description] - COMPLETE

**Input:** [Describe input state]

**Operation:** [What was done]

**Output:**
- Shape: [final shape]
- Key findings: [observations]

**Verification:** [How you confirmed it worked]

**Next:** [What comes next]

Task Summary (MANDATORY after each task)

After a task passes review, append a structured summary to LEARNINGS.md:

## Task N: [task description]

---
task: N
status: completed
implements: [DATA-01, STAT-03]
affects: [notebooks/analysis.py, data/processed/]
key-files:
  created: [list of new files]
  modified: [list of changed files]
deviations: {r1: 0, r2: 1, r3: 0, r4: 0}
---

One-liner: [SUBSTANTIVE summary — not "Task complete" but "Merged CRSP-Compustat panel with winsorized returns at 1%/99%"]

Changes: [what was added/modified and why]
Output: [output files produced and their contents]

One-liner rule: Must be SUBSTANTIVE. Good: "Panel regression with firm and year FE, clustered SEs, 3 robustness checks". Bad: "Completed task 3".

Verification Patterns

See references/verification-patterns.md for detailed code patterns for:

• Data loading, filtering, merging
• Aggregation and model training
• Batch pipeline scale-up testing (submission, validation, cost extrapolation)
• Quick reference table by operation type

See references/etl-enforcement.md for ETL strategy enforcement:

• Filter push-down (database vs application vs hybrid)
• Parallelism (Task agents vs SGE vs sequential)
• Intermediate caching (parquet vs CSV vs SQLite)
• Scale-up testing domain routing

Scale-Up Testing Protocol (Batch/ETL Operations)

Triggers when PLAN.md includes a Scale-Up Testing Plan table. NO FULL BATCH WITHOUT A SUCCESSFUL TEST BATCH. This is not negotiable.

Three stages: Test (~10 items, always required) -> Intermediate (~100, if total >500) -> Large (~1,000, if total >5,000). Each stage has quality gates that must pass before scaling up.

Full protocol, rationalization table, and red flags: See references/scale-up-testing.md

Common Failures to Avoid

If Output Looks Wrong

1. STOP - do not proceed
2. Investigate - print more details
3. Document - log the issue in LEARNINGS.md
4. Ask - if unclear, ask user for guidance
5. Fix - only proceed after output verified

Never hide failures. Bad output documented is better than silent failure.

No Pause Between Tasks

Your pausing between tasks is procrastination disguised as courtesy. </EXTREMELY-IMPORTANT>

Deviation Rules

When subagents encounter unplanned issues during implementation, follow this 4-rule system:

Priority: R4a/R4b (STOP) > R1-R3 (auto) > unsure → escalate as R4.

Edge cases:

• Unexpected nulls in non-key column → R2 (add handling)
• Unexpected nulls in key/ID column → R4a (data assumption violated)
• Package version mismatch → R3 (blocking)
• Need different statistical test → R4b (methodology change)
• Wrong merge type (left vs inner) → R1 (bug)
• Data has different granularity than expected → R4a (assumption)

Tracking format per task: Each task summary in .planning/LEARNINGS.md should end with: Deviations: N auto-fixed (R1: X, R2: Y, R3: Z). R4 escalations: [list or "none"].

Agent Team Implementation (Parallel)

Full protocol: See references/agent-team-protocol.md for prerequisites, spawn prompt template, lead monitoring, reconciliation (3 passes), and usage guidelines.

Key points:

1. Run foundation tasks sequentially FIRST, then spawn parallel teammates
2. Each teammate gets exclusive data scope and output files
3. After all teammates complete, lead performs 3 reconciliation passes (Collect, Verify, Methodology)
4. Default is sequential. Only use agent teams for 4+ tasks with true independence (different datasets/subsets, no shared output files)

Gate: Exit Implementation

Checkpoint type: human-verify (all tasks pass — machine-verifiable)

Before proceeding to validation, execute this gate:

1. IDENTIFY: Read .planning/PLAN.md — list every task by number and name
2. RUN: Read .planning/LEARNINGS.md — find entries for each task
3. READ: For each task, confirm LEARNINGS.md contains:
   • A "Task N: [Name] - COMPLETE" entry
   • Verified output (shape, stats, or sample)
   • No unresolved issues flagged
4. VERIFY: Count tasks in PLAN.md vs completed entries in LEARNINGS.md. They MUST match.

Staleness Check: LEARNINGS.md must be updated in THIS session, not reused from prior work.

• Does each task entry reference current outputs (file paths, cell numbers)?
• If LEARNINGS.md is stale from a prior session, UPDATE it with fresh entries before claiming completion.

Stale LEARNINGS.md = false gate pass = unverified work = the user gets results no one actually checked.

5. CLAIM: Only if all tasks accounted for, proceed to review

If ANY task is missing from LEARNINGS.md, implement it before proceeding.

Claiming all tasks are done without checking LEARNINGS.md against PLAN.md is NOT HELPFUL — missing tasks mean incomplete analysis the user relies on. </EXTREMELY-IMPORTANT>

Phase Complete

After passing the exit gate, IMMEDIATELY discover and read the validation phase: Read ${CLAUDE_SKILL_DIR}/../../skills/ds-validate/SKILL.md and follow its instructions. Follow its instructions to validate outputs before review.