Fixing Tests

Surgical test remediation. Three input modes, phased execution, verified output.

Invariant Principles

1. Tests catch bugs, not green checkmarks. Every fix must detect real failures, not just pass.
2. Production bugs are not test issues. Flag and escalate; never silently "fix" broken behavior.
3. Read before fixing. Never guess at code structure or blindly apply suggestions.
4. Verify proves value. Unverified fixes are unfinished fixes.
5. Scope discipline. Fix tests, not features. No over-engineering, no under-testing.

Inputs

One of the three input modes required. If unclear, ask user to clarify target.

Outputs

Input Mode Detection

If unclear: ask user to clarify target.

Phase 0: Input Processing

Build WorkItem list:

interface WorkItem {
  id: string;
  priority: "critical" | "important" | "minor" | "unknown";
  test_file: string;
  test_function?: string;
  pattern?: number;           // 1-8 from green mirage
  blind_spot?: string;        // What broken code would pass
  error_message?: string;     // For run_and_fix mode
}

Optional: ask commit strategy (per-fix recommended, batch-by-file, single).

Phase 1: Discovery (run_and_fix only)

Skip for audit_report/general_instructions modes.

Run test suite, parse failures into WorkItems with error_type, message, stack trace.

Phase 2: Fix Execution

Process by priority: critical > important > minor.

For EACH work item:

2.1 Investigate

Read test file + production code. Audit suggestions are starting points, not gospel.

2.2 Classify Fix Type

2.3 Production Bug Protocol

PRODUCTION BUG DETECTED
Test: [test_function]
Expected: [what test expects]
Actual: [what code does]

Options:
A) Fix production bug (test will pass)
B) Update test to match buggy behavior (not recommended)
C) Skip test, create issue

Do NOT silently fix production bugs as "test fixes." </CRITICAL>

2.4 Apply and Verify

# Run fixed test
pytest path/to/test.py::test_function -v

# Check file for side effects
pytest path/to/test.py -v

Verification:

• Specific test passes
• File tests still pass
• Fix would catch actual failure

2.5 Commit (if per-fix strategy)

fix(tests): strengthen assertions in test_function

- [What was weak/broken]
- [What fix does]
- Pattern: N - [name] (if from audit)

Phase 3: Batch Processing

FOR priority IN [critical, important, minor]:
  FOR item IN work_items[priority]:
    Execute Phase 2
    IF stuck after 2 attempts: add to stuck_items, continue

Phase 4: Final Verification

Run full test suite. Report:

Include stuck items with recommendations, production bugs with actions.

Special Cases

Flaky tests: Identify non-determinism source, mock/control it. Use deterministic waits, not sleep-and-hope.

Implementation-coupled tests: Rewrite to test behavior through public interface.

Missing tests entirely: Read production code, identify key behaviors, write tests following codebase patterns.

Green Mirage Audit Integration

Parse YAML block between --- markers. Use remediation_plan.phases for execution order. Honor depends_on dependencies. Batch by file when possible.

Fallback: parse legacy markdown format by **Finding #N:** headers.

Self-Check

Before completing:

• All items processed or marked stuck
• Each fix verified to pass
• Each fix verified to catch failure it should
• Full suite ran at end
• Production bugs flagged, not silently fixed
• Commits follow strategy
• Summary provided

If ANY unchecked: STOP and fix.

<FINAL_EMPHASIS> Fix it. Prove it works. Move on. No over-engineering. No under-testing. </FINAL_EMPHASIS>