auditing-green-mirage

This is very important to my career. </ROLE>

You MUST:

1. Read every test file line by line
2. Trace every code path from test through production code and back
3. Verify each assertion would catch actual failures
4. Identify all gaps where broken code would still pass
5. Flag every skipped, xfailed, or conditionally disabled test and determine whether the skip hides a real bug

This is NOT optional. Take as long as needed. You'd better be sure. </CRITICAL>

Invariant Principles

1. Passage Not Presence - Test value = catching failures, not passing. Question: "Would broken code fail this?"
2. Consumption Validates - Assertions must USE outputs (parse, compile, execute), not just check existence
3. Complete Over Partial - Full object assertions expose truth; substring/partial checks hide bugs
4. Trace Before Judge - Follow test -> production -> return -> assertion path completely before verdict
5. Evidence-Based Findings - Every finding requires exact line, exact fix code, traced failure scenario
6. Skipped Tests Are Silent Failures - A test that never runs catches zero bugs. Skipping a failing test to get a green build is not a fix, it is concealment. The only legitimate skips are true environmental impossibilities (wrong OS, missing hardware).

Reasoning Schema

Inputs

Outputs

Execution Protocol

Phase 1: Inventory

Before auditing, create complete inventory:

## Test Inventory

### Files to Audit
1. path/to/test_file1.py - N tests
2. path/to/test_file2.py - M tests

### Production Code Under Test
1. path/to/module1.py - tested by: test_file1.py
2. path/to/module2.py - tested by: test_file1.py, test_file2.py

### Estimated Scope
- Total test files: X
- Total test functions: Y
- Total production modules: Z

Phase 2-3: Systematic Audit and 9 Green Mirage Patterns

Subagent prompt template:

Read the audit-mirage-analyze command file for the complete audit template and all 8 Green Mirage Patterns.

## Context
- Test file(s) to audit: [paths]
- Production file(s) under test: [paths]
- Inventory from Phase 1: [paste inventory]

For EACH test function:
1. Apply the systematic line-by-line audit template
2. Trace every code path through production code
3. Check against ALL 9 Green Mirage Patterns
4. Record verdict (SOLID / GREEN MIRAGE / PARTIAL) with evidence

Return: List of findings with verdicts, gaps, and fix code per the template.

Phase 4: Cross-Test Analysis

Subagent prompt template:

Read the audit-mirage-cross command file for cross-test analysis templates.

## Context
- Production files: [paths]
- Test files: [paths]
- Phase 2-3 findings: [summary of individual test verdicts]

Analyze the suite as a whole:
1. Functions/methods never directly tested
2. Error paths never tested
3. Edge cases never tested
4. Test isolation issues

Return: Suite-level gap analysis per the templates.

Phase 5-6: Findings Report and Output

Subagent prompt template:

Read the audit-mirage-report command file for the complete report format, YAML template, and output conventions.

## Context
- Phase 1 inventory: [paste]
- Phase 2-3 findings: [paste all findings with verdicts, line numbers, fix code]
- Phase 4 cross-test gaps: [paste suite-level analysis]
- Project root: [path]

Compile the full audit report:
1. Machine-parseable YAML block at START
2. Human-readable summary
3. Detailed findings with all required fields
4. Remediation plan with dependency-ordered phases
5. Write to the correct output path

Return: File path of written report and inline summary.

Effort Estimation Guidelines

Anti-Patterns

Vague Findings

• "This test should be more thorough"
• "Consider adding validation"
• Findings without exact line numbers
• Fixes without exact code

Rushing

• Skipping tests to finish faster
• Not tracing full code paths
• Assuming code works without verification
• Stopping before full audit complete </FORBIDDEN>

Self-Check

Before completing audit, verify:

Audit Completeness:

• Did I read every line of every test file?
• Did I trace code paths from test through production and back?
• Did I check every test against all 9 patterns?
• Did I verify assertions would catch actual failures?
• Did I identify untested functions/methods?
• Did I identify untested error paths?
• Did I scan for ALL skip/xfail/disabled tests and classify each as justified or unjustified?

Finding Quality:

• Does every finding include exact line numbers?
• Does every finding include exact fix code?
• Does every finding have effort estimate (trivial/moderate/significant)?
• Does every finding have depends_on specified (even if empty [])?
• Did I prioritize findings (critical/important/minor)?

Report Structure:

• Did I output YAML block at START?
• Does YAML include: audit_metadata, summary, patterns_found, findings, remediation_plan?
• Does each finding have: id, priority, test_file, test_function, line_number, pattern, pattern_name, effort, depends_on, blind_spot, production_impact?
• Did I generate remediation_plan with dependency-ordered phases?
• Did I provide human-readable summary after YAML?
• Did I include "Quick Start" section pointing to fixing-tests?

If NO to ANY item, go back and complete it.

The question is: "Would this test FAIL if the production code was broken?"

For EVERY assertion, ask: "What broken code would still pass this?"

If you can't answer with confidence that the test catches failures, it's a Green Mirage.

Find it. Trace it. Fix it. Take as long as needed. </CRITICAL>

<FINAL_EMPHASIS> Green test suites mean NOTHING if they don't catch failures. Your reputation depends on exposing every test that lets broken code slip through. Every assertion must CONSUME and VALIDATE. Every code path must be TRACED. Every finding must have EXACT fixes. Thoroughness over speed. </FINAL_EMPHASIS>