Agent skill

test-strategy

Test pyramid decision matrix, coverage targets, when to write which test type, mock vs real dependency decisions, and test ROI analysis.

View SKILL.md on GitHub Repository

Stars 458

Forks 38

Install this agent skill to your Project

npx add-skill https://github.com/vibeeval/vibecosystem/tree/main/skills/test-strategy

SKILL.md

Test Strategy

Test Pyramid Ratio Guidance

        /\
       /e2e\         ~10% — critical user flows only
      /------\
     /  integ  \     ~20% — API contracts, DB interactions
    /------------\
   /    unit      \  ~70% — pure logic, transformations, edge cases
  /--------------\

Keep the pyramid right-side-up. Inverting it (too many e2e) leads to slow, flaky CI.

Decision Matrix: Task Type → Test Type

Task Type	Test Type	Tool
Pure function / utility	Unit	Jest / Vitest
API endpoint	Integration	Supertest / httpx
Critical user flow	E2E	Playwright
Data transformation	Property-based	fast-check / Hypothesis
React/Vue component	Component	Testing Library
CLI command	Integration	execa + assertions
Database query	Integration (real DB)	jest + pg / pytest
Cron job / scheduler	Unit (mocked time)	Jest fakeTimers

Mock vs Real Dependency Decision Tree

Is it an external API (Stripe, Sendgrid, etc.)?
  → YES: Always mock. Use recorded fixtures or MSW.

Is it a database?
  → Unit test context: mock (in-memory store or jest.fn())
  → Integration test context: real DB (test container or local)

Is it the file system?
  → Mock with memfs or tmp dir, then clean up.

Is it time / Date.now()?
  → Always mock. Use Jest fakeTimers or freezegun (Python).

Is it a third-party SDK wrapper you wrote?
  → Skip testing the wrapper itself, test your code's behavior.

Coverage Targets by Project Type

Project Type	Branch Coverage	Notes
Published library	90%+	Every exported function needs tests
Production app	80%+	Focus on critical paths
Internal tool	70%+	Happy path + main error cases
Prototype / spike	Skip	Throw it away anyway
Generated code	Skip	Don't test codegen output

Test Naming Conventions

Jest / Vitest (describe + it)

typescript

describe('calculateDiscount', () => {
  it('returns 10% for gold members', () => { ... })
  it('returns 0% when cart is empty', () => { ... })
  it('throws when discount rate exceeds 100', () => { ... })
})

Given-When-Then (BDD style)

typescript

describe('OrderService', () => {
  describe('given a confirmed order', () => {
    describe('when the user cancels', () => {
      it('then it transitions to CANCELLED state', () => { ... })
      it('then it sends a cancellation email', () => { ... })
    })
  })
})

When NOT to Test

Generated code (Prisma client, GraphQL types, protobuf outputs)
Third-party SDK wrappers with zero custom logic
Trivial getters/setters (getEmail() { return this.email })
Config files
Framework boilerplate (Next.js _app.tsx, Express server bootstrap)

Test Isolation Strategies

Transaction rollback (PostgreSQL)

typescript

beforeEach(async () => {
  await db.query('BEGIN')
})

afterEach(async () => {
  await db.query('ROLLBACK')
})

Cleanup hooks

typescript

afterEach(() => {
  jest.clearAllMocks()        // clear call counts
  jest.resetAllMocks()        // reset return values
  jest.restoreAllMocks()      // restore spied originals
})

Test containers (real DB, isolated)

typescript

import { PostgreSqlContainer } from '@testcontainers/postgresql'

let container: StartedPostgreSqlContainer

beforeAll(async () => {
  container = await new PostgreSqlContainer().start()
  process.env.DATABASE_URL = container.getConnectionUri()
})

afterAll(async () => {
  await container.stop()
})

Flaky Test Triage

When a test is flaky (passes/fails non-deterministically):

Check for shared mutable state (global variables, singleton caches)
Check for missing await on async calls
Check for time-dependent assertions (setTimeout, Date.now())
Check for ordering dependencies (tests relying on previous test state)
Add --runInBand to isolate and confirm

Mutation Testing (Stryker)

Mutation testing verifies that your tests actually catch bugs:

bash

npx stryker run

json

// stryker.config.json
{
  "mutator": { "excludedMutations": ["StringLiteral"] },
  "thresholds": { "high": 80, "low": 60, "break": 50 },
  "reporters": ["html", "progress"]
}

Mutation score < 60% means tests pass without catching real logic errors. Focus on the surviving mutants — each one is an untested code path.