Agent skill

characterization-test-generator

Generate characterization tests to capture and verify existing behavior before migration

View SKILL.md on GitHub Repository
Stars 514
Forks 31

Install this agent skill to your Project

npx add-skill https://github.com/a5c-ai/babysitter/tree/main/library/specializations/code-migration-modernization/skills/characterization-test-generator

SKILL.md

Characterization Test Generator Skill

Generates characterization tests (also known as golden master tests or approval tests) to capture existing system behavior before migration, ensuring functional equivalence after changes.

Purpose

Enable behavior preservation during migration through:

  • Golden master test creation
  • Approval test generation
  • Edge case discovery
  • Input/output recording
  • Behavior snapshot capture
  • Regression baseline establishment

Capabilities

1. Golden Master Test Creation

  • Capture current system outputs for given inputs
  • Store outputs as reference "golden" files
  • Generate comparison test infrastructure
  • Support multiple output formats
  • Enable baseline versioning

2. Approval Test Generation

  • Create human-reviewable test outputs
  • Generate diff-based comparison tests
  • Support approval workflows
  • Enable incremental approval
  • Track approval history

3. Edge Case Discovery

  • Analyze code paths for boundary conditions
  • Generate boundary value inputs
  • Identify error handling scenarios
  • Map exception paths
  • Create comprehensive test coverage

4. Input/Output Recording

  • Instrument code for I/O capture
  • Record API request/response pairs
  • Capture database interactions
  • Log external service calls
  • Store timing and sequence data

5. Behavior Snapshot

  • Capture state at key points
  • Record side effects
  • Document non-deterministic behavior
  • Identify environmental dependencies
  • Map configuration impacts

6. Regression Baseline Establishment

  • Create baseline test suites
  • Define acceptance thresholds
  • Configure tolerance levels
  • Set up CI/CD integration
  • Generate coverage reports

Tool Integrations

This skill can leverage the following external tools when available:

Tool Purpose Integration Method
ApprovalTests Approval testing framework Library
Jest snapshots JavaScript snapshot testing Framework
pytest-snapshot Python snapshot testing Plugin
TextTest Golden master testing CLI
Verify .NET approval testing Library
Scientist Safe refactoring library Library
AI Testing MCP AI-powered test generation MCP Server

Usage

Basic Test Generation

bash
# Invoke skill for characterization test generation
# The skill will analyze code and generate tests

# Expected inputs:
# - targetPath: Path to code to characterize
# - testFramework: 'jest' | 'pytest' | 'junit' | 'nunit' | 'auto'
# - outputDir: Directory for generated tests
# - captureMode: 'snapshot' | 'approval' | 'recording'

Generation Workflow

  1. Analysis Phase

    • Identify testable units (functions, methods, endpoints)
    • Map input parameters and types
    • Detect output formats
    • Find external dependencies

  2. Input Discovery Phase

    • Extract existing test inputs
    • Generate boundary values
    • Identify representative cases
    • Create combinatorial inputs

  3. Capture Phase

    • Execute code with inputs
    • Record outputs and side effects
    • Capture state changes
    • Log external interactions

  4. Test Generation Phase

    • Generate test code structure
    • Create golden master files
    • Set up comparison logic
    • Configure CI integration

Output Schema

json
{
  "generationId": "string",
  "timestamp": "ISO8601",
  "target": {
    "path": "string",
    "language": "string",
    "framework": "string",
    "unitsAnalyzed": "number"
  },
  "testsGenerated": {
    "total": "number",
    "byType": {
      "snapshot": "number",
      "approval": "number",
      "recording": "number"
    },
    "coverage": {
      "functions": "number",
      "branches": "number",
      "lines": "number"
    }
  },
  "characterizations": [
    {
      "unit": "string",
      "type": "function|method|endpoint|class",
      "inputs": [
        {
          "name": "string",
          "type": "string",
          "values": ["any"],
          "source": "existing|generated|boundary"
        }
      ],
      "outputs": {
        "type": "string",
        "goldenMasterPath": "string",
        "checksum": "string"
      },
      "testFile": "string",
      "baselineApproved": "boolean"
    }
  ],
  "edgeCases": [
    {
      "unit": "string",
      "case": "string",
      "input": "any",
      "expectedBehavior": "string",
      "covered": "boolean"
    }
  ],
  "dependencies": {
    "external": ["string"],
    "mocked": ["string"],
    "recorded": ["string"]
  },
  "artifacts": {
    "testSuite": "string",
    "goldenMasters": "string",
    "recordings": "string",
    "coverageReport": "string"
  }
}

Integration with Migration Processes

This skill integrates with the following Code Migration/Modernization processes:

  • migration-testing-strategy: Primary tool for test baseline creation
  • code-refactoring: Ensure behavior preservation during refactoring
  • framework-upgrade: Verify functionality after upgrade
  • monolith-to-microservices: Validate service extraction

Configuration

Skill Configuration File

Create .characterization-tests.json in the project root:

json
{
  "testFramework": "auto",
  "outputDir": "./tests/characterization",
  "captureMode": "snapshot",
  "goldenMasterDir": "./tests/golden-masters",
  "inputGeneration": {
    "boundary": true,
    "combinatorial": true,
    "maxCombinations": 100,
    "includeNulls": true,
    "includeEmpty": true
  },
  "outputCapture": {
    "format": "json",
    "normalizeWhitespace": true,
    "ignorePaths": ["$.timestamp", "$.requestId"],
    "tolerance": {
      "numeric": 0.001,
      "dateTime": "1s"
    }
  },
  "dependencies": {
    "mockExternal": true,
    "recordMode": false,
    "replayMode": true
  },
  "approval": {
    "requireApproval": true,
    "approvalDir": "./tests/approvals",
    "reportFormat": "markdown"
  },
  "ci": {
    "failOnNewTests": false,
    "updateGoldenOnPass": false,
    "generateCoverageReport": true
  }
}

MCP Server Integration

When AI Testing MCP Server is available:

javascript
// Example AI-powered test generation
{
  "tool": "ai_testing_generate",
  "arguments": {
    "target": "./src/services/user.ts",
    "framework": "jest",
    "style": "characterization"
  }
}

Test Patterns

Snapshot Testing (Jest)

javascript
// Generated characterization test
describe('UserService', () => {
  describe('calculateDiscount', () => {
    it('should match snapshot for standard customer', () => {
      const result = userService.calculateDiscount({
        customerId: 'C001',
        purchaseAmount: 100,
        loyaltyPoints: 500
      });
      expect(result).toMatchSnapshot();
    });

    it('should match snapshot for premium customer', () => {
      const result = userService.calculateDiscount({
        customerId: 'C002',
        purchaseAmount: 100,
        loyaltyPoints: 5000,
        isPremium: true
      });
      expect(result).toMatchSnapshot();
    });

    // Edge cases
    it('should match snapshot for zero amount', () => {
      const result = userService.calculateDiscount({
        customerId: 'C001',
        purchaseAmount: 0,
        loyaltyPoints: 0
      });
      expect(result).toMatchSnapshot();
    });
  });
});

Approval Testing (ApprovalTests)

java
// Generated approval test
public class UserServiceCharacterizationTest {

    @Test
    public void calculateDiscount_standardCustomer() {
        UserService service = new UserService();
        DiscountResult result = service.calculateDiscount(
            new DiscountRequest("C001", 100.0, 500)
        );
        Approvals.verify(result);
    }

    @Test
    public void calculateDiscount_boundaryValues() {
        UserService service = new UserService();

        // Boundary: minimum values
        Approvals.verify(service.calculateDiscount(
            new DiscountRequest("C001", 0.01, 0)
        ), "minimum");

        // Boundary: maximum values
        Approvals.verify(service.calculateDiscount(
            new DiscountRequest("C001", 999999.99, 999999)
        ), "maximum");
    }
}

Recording Tests (Python)

python
# Generated recording-based test
import pytest
from tests.recordings import PlaybackRecorder

class TestUserServiceCharacterization:

    @pytest.fixture
    def recorder(self):
        return PlaybackRecorder('tests/recordings/user_service')

    def test_get_user_profile_recorded(self, recorder):
        """Replay recorded external API interactions"""
        with recorder.playback('get_user_profile_c001'):
            service = UserService()
            result = service.get_user_profile('C001')

            assert result == recorder.expected_output()

    def test_update_user_settings_recorded(self, recorder):
        """Verify database interactions match recording"""
        with recorder.playback('update_settings_c001'):
            service = UserService()
            result = service.update_settings('C001', {'theme': 'dark'})

            recorder.verify_database_calls()
            assert result == recorder.expected_output()

Edge Case Categories

Boundary Values

  • Minimum/maximum values
  • Zero values
  • Empty strings/arrays
  • Null/undefined values
  • Type boundaries (INT_MAX, etc.)

Error Conditions

  • Invalid inputs
  • Missing required fields
  • Type mismatches
  • Constraint violations
  • Authentication failures

Concurrent Scenarios

  • Race conditions
  • Deadlocks
  • Timeout scenarios
  • Partial failures
  • Retry behaviors

Environmental

  • Configuration variations
  • Timezone differences
  • Locale changes
  • Feature flag states
  • Permission levels

Best Practices

  1. Start Before Changes: Generate characterization tests before any migration work
  2. Capture Everything: Record all outputs, side effects, and state changes
  3. Version Golden Masters: Store golden masters in version control
  4. Review Approvals: Human review of initial golden masters is essential
  5. Handle Non-Determinism: Normalize timestamps, IDs, and random values
  6. Incremental Updates: Update baselines incrementally as changes are approved
  7. CI Integration: Fail builds on unexpected behavior changes

Related Skills

  • test-coverage-analyzer: Analyze coverage gaps
  • migration-validator: Validate migration results
  • static-code-analyzer: Identify testable code paths

Related Agents

  • migration-testing-strategist: Uses this skill for test strategy
  • regression-detector: Uses this skill for regression detection
  • parallel-run-validator: Uses this skill for comparison testing

References

Recommended Agent Skills

Expand your agent's capabilities with these related and highly-rated skills.

a5c-ai/babysitter

gsd-tools

Central utility skill for GSD operations. Provides config parsing, slug generation, timestamps, path operations, and orchestrates calls to other specialized skills. Acts as the unified entry point that the original gsd-tools.cjs provided via its lib/ modules (commands, config, core, init).

514 31
Explore
a5c-ai/babysitter

model-profile-resolution

Resolve model profile (quality/balanced/budget) at orchestration start and map agents to specific models. Enables cost/quality tradeoffs by selecting appropriate AI models for each agent role.

514 31
Explore
a5c-ai/babysitter

verification-suite

Plan structure validation, phase completeness checks, reference integrity verification, and artifact existence confirmation. Provides the structured verification layer ensuring GSD artifacts are well-formed and complete.

514 31
Explore
a5c-ai/babysitter

state-management

STATE.md reading, writing, and field-level updates. Provides cross-session state persistence via .planning/STATE.md with structured fields for current task, completed phases, blockers, decisions, and quick tasks.

514 31
Explore
a5c-ai/babysitter

git-integration

Git commit patterns, formats, and conventions for GSD methodology. Provides atomic commits per task, structured commit messages, planning file commits, branch management, and milestone tag operations.

514 31
Explore
a5c-ai/babysitter

frontmatter-parsing

YAML frontmatter parsing and manipulation for .planning/ documents. Provides read, write, update, query, and validation operations on frontmatter blocks in GSD markdown artifacts.

514 31
Explore

Didn't find tool you were looking for?

Be as detailed as possible for better results