Agent skill

property-based-test-generator

Generates property-based tests using Hypothesis (Python), fast-check (JavaScript/TypeScript), or QuickCheck (Haskell). Use when user asks to "generate property tests", "create hypothesis tests", "add property-based testing", or "generate fast-check tests".

View SKILL.md on GitHub Repository
Stars 5
Forks 2

Install this agent skill to your Project

npx add-skill https://github.com/Dexploarer/claudius-skills/tree/main/examples/intermediate/testing-skills/property-based-test-generator

SKILL.md

Property-Based Test Generator

Generates property-based tests that validate invariants and find edge cases automatically through randomized testing.

When to Use

  • "Generate property-based tests"
  • "Create hypothesis tests for my function"
  • "Add property tests to my code"
  • "Generate fast-check tests"
  • "Test function properties"
  • "Find edge cases automatically"

Instructions

1. Detect Language and Testing Framework

Check the project's language and existing test setup:

bash
# Check for Python
[ -f "pytest.ini" ] || [ -f "setup.py" ] && echo "Python"

# Check for JavaScript/TypeScript
[ -f "package.json" ] && echo "JavaScript/TypeScript"

# Check existing test framework
grep -E "(jest|vitest|mocha|pytest|hypothesis)" package.json pyproject.toml requirements.txt 2>/dev/null

2. Install Property-Based Testing Library

Python (Hypothesis):

bash
pip install hypothesis pytest

JavaScript/TypeScript (fast-check):

bash
npm install --save-dev fast-check @types/jest
# or
npm install --save-dev fast-check vitest

Haskell (QuickCheck):

bash
cabal install QuickCheck

3. Identify Function Properties

Analyze the function to test and identify invariants:

Common Properties:

  • Idempotence: f(f(x)) === f(x)
  • Inverse: decode(encode(x)) === x
  • Commutativity: f(a, b) === f(b, a)
  • Associativity: f(f(a, b), c) === f(a, f(b, c))
  • Identity: f(x, identity) === x
  • Range: Output always within valid range
  • Type safety: Output type matches expected
  • No exceptions: Function never throws for valid input

4. Generate Property-Based Tests

Python with Hypothesis

Basic Example:

python
from hypothesis import given, strategies as st
import pytest

# Function to test
def sort_list(items):
    return sorted(items)

# Property: sorted list length equals original
@given(st.lists(st.integers()))
def test_sort_preserves_length(items):
    result = sort_list(items)
    assert len(result) == len(items)

# Property: sorted list is ordered
@given(st.lists(st.integers()))
def test_sort_creates_ordered_list(items):
    result = sort_list(items)
    for i in range(len(result) - 1):
        assert result[i] <= result[i + 1]

# Property: sorted list contains same elements
@given(st.lists(st.integers()))
def test_sort_preserves_elements(items):
    result = sort_list(items)
    assert sorted(items) == result

Advanced Strategies:

python
from hypothesis import given, strategies as st, assume
from datetime import datetime, timedelta

# Custom data structures
@st.composite
def users(draw):
    return {
        'id': draw(st.integers(min_value=1, max_value=1000000)),
        'name': draw(st.text(min_size=1, max_size=50)),
        'email': draw(st.emails()),
        'age': draw(st.integers(min_value=18, max_value=120)),
        'created_at': draw(st.datetimes(
            min_value=datetime(2020, 1, 1),
            max_value=datetime.now()
        ))
    }

@given(users())
def test_user_validation(user):
    # Validate user properties
    assert user['age'] >= 18
    assert '@' in user['email']
    assert len(user['name']) > 0
    assert user['created_at'] <= datetime.now()

Testing with Preconditions:

python
@given(st.integers(), st.integers())
def test_division(a, b):
    assume(b != 0)  # Precondition: no division by zero
    result = a / b
    assert result * b == a  # Property: inverse of multiplication

Stateful Testing:

python
from hypothesis.stateful import RuleBasedStateMachine, rule, invariant

class ShoppingCart(RuleBasedStateMachine):
    def __init__(self):
        super().__init__()
        self.items = []
        self.total = 0

    @rule(item=st.tuples(st.text(), st.floats(min_value=0, max_value=1000)))
    def add_item(self, item):
        name, price = item
        self.items.append(item)
        self.total += price

    @rule()
    def clear_cart(self):
        self.items = []
        self.total = 0

    @invariant()
    def total_matches_sum(self):
        assert abs(self.total - sum(p for _, p in self.items)) < 0.01

TestCart = ShoppingCart.TestCase

JavaScript/TypeScript with fast-check

Basic Example:

typescript
import fc from 'fast-check';

// Function to test
function reverseString(str: string): string {
  return str.split('').reverse().join('');
}

describe('reverseString', () => {
  it('double reverse returns original', () => {
    fc.assert(
      fc.property(fc.string(), (str) => {
        const reversed = reverseString(str);
        const doubleReversed = reverseString(reversed);
        return doubleReversed === str;
      })
    );
  });

  it('preserves string length', () => {
    fc.assert(
      fc.property(fc.string(), (str) => {
        return reverseString(str).length === str.length;
      })
    );
  });

  it('first char becomes last char', () => {
    fc.assert(
      fc.property(fc.string({ minLength: 1 }), (str) => {
        const reversed = reverseString(str);
        return str[0] === reversed[reversed.length - 1];
      })
    );
  });
});

Complex Data Structures:

typescript
import fc from 'fast-check';

// Custom arbitraries
const userArbitrary = fc.record({
  id: fc.integer({ min: 1, max: 1000000 }),
  name: fc.string({ minLength: 1, maxLength: 50 }),
  email: fc.emailAddress(),
  age: fc.integer({ min: 18, max: 120 }),
  roles: fc.array(fc.constantFrom('admin', 'user', 'guest'), { minLength: 1 })
});

describe('User validation', () => {
  it('validates user structure', () => {
    fc.assert(
      fc.property(userArbitrary, (user) => {
        return (
          user.age >= 18 &&
          user.email.includes('@') &&
          user.roles.length > 0
        );
      })
    );
  });
});

Array Properties:

typescript
describe('Array operations', () => {
  it('map preserves length', () => {
    fc.assert(
      fc.property(
        fc.array(fc.integer()),
        fc.func(fc.integer()),
        (arr, fn) => {
          return arr.map(fn).length === arr.length;
        }
      )
    );
  });

  it('filter result is subset', () => {
    fc.assert(
      fc.property(
        fc.array(fc.integer()),
        (arr) => {
          const filtered = arr.filter(x => x > 0);
          return filtered.every(x => arr.includes(x));
        }
      )
    );
  });

  it('concat is associative', () => {
    fc.assert(
      fc.property(
        fc.array(fc.integer()),
        fc.array(fc.integer()),
        fc.array(fc.integer()),
        (a, b, c) => {
          const left = a.concat(b).concat(c);
          const right = a.concat(b.concat(c));
          return JSON.stringify(left) === JSON.stringify(right);
        }
      )
    );
  });
});

Shrinking Examples:

typescript
describe('Shrinking demonstration', () => {
  it('finds minimal failing case', () => {
    fc.assert(
      fc.property(fc.array(fc.integer()), (arr) => {
        // This will fail and shrink to smallest failing case
        return arr.length < 5 || arr.some(x => x > 100);
      }),
      { numRuns: 100 }
    );
  });
});

Model-Based Testing:

typescript
import fc from 'fast-check';

class Stack<T> {
  private items: T[] = [];

  push(item: T): void {
    this.items.push(item);
  }

  pop(): T | undefined {
    return this.items.pop();
  }

  size(): number {
    return this.items.length;
  }
}

describe('Stack', () => {
  it('behaves like array', () => {
    fc.assert(
      fc.property(
        fc.array(fc.integer()),
        (operations) => {
          const stack = new Stack<number>();
          const model: number[] = [];

          for (const op of operations) {
            if (op >= 0) {
              stack.push(op);
              model.push(op);
            } else {
              const stackResult = stack.pop();
              const modelResult = model.pop();
              if (stackResult !== modelResult) return false;
            }
          }

          return stack.size() === model.length;
        }
      )
    );
  });
});

5. Common Property Patterns

Encode/Decode (Roundtrip):

python
from hypothesis import given, strategies as st
import json

@given(st.dictionaries(st.text(), st.integers()))
def test_json_roundtrip(data):
    encoded = json.dumps(data)
    decoded = json.loads(encoded)
    assert decoded == data
typescript
fc.assert(
  fc.property(fc.anything(), (data) => {
    const encoded = JSON.stringify(data);
    const decoded = JSON.parse(encoded);
    return JSON.stringify(decoded) === encoded;
  })
);

Idempotence:

python
@given(st.lists(st.integers()))
def test_dedup_idempotent(items):
    result1 = list(set(items))
    result2 = list(set(result1))
    assert result1 == result2

Commutativity:

python
@given(st.integers(), st.integers())
def test_addition_commutative(a, b):
    assert a + b == b + a

Oracle (Compare with Known Implementation):

python
@given(st.lists(st.integers()))
def test_custom_sort_matches_builtin(items):
    assert custom_sort(items) == sorted(items)

Invariants:

typescript
fc.assert(
  fc.property(fc.array(fc.integer()), (arr) => {
    const unique = [...new Set(arr)];
    return unique.length <= arr.length;
  })
);

6. Configuration Options

Hypothesis:

python
from hypothesis import given, settings, strategies as st

@settings(
    max_examples=1000,  # Number of test cases
    deadline=None,      # No timeout
    verbosity=hypothesis.Verbosity.verbose
)
@given(st.integers())
def test_with_settings(x):
    assert x == x

fast-check:

typescript
fc.assert(
  fc.property(fc.integer(), (x) => x === x),
  {
    numRuns: 1000,        // Number of test cases
    seed: 42,             // Reproducible tests
    verbose: true,        // Show details
    endOnFailure: false   // Run all tests
  }
);

7. Testing Strategies by Data Type

Strings:

python
st.text()
st.text(min_size=1, max_size=100)
st.text(alphabet=st.characters(blacklist_categories=['Cs']))
st.from_regex(r'[a-z]{3,10}')
typescript
fc.string()
fc.string({ minLength: 1, maxLength: 100 })
fc.hexaString()
fc.asciiString()
fc.unicodeString()
fc.stringOf(fc.char())

Numbers:

python
st.integers()
st.integers(min_value=0, max_value=100)
st.floats(min_value=0.0, max_value=1.0)
st.decimals()
typescript
fc.integer()
fc.integer({ min: 0, max: 100 })
fc.float()
fc.double()
fc.nat()

Collections:

python
st.lists(st.integers())
st.lists(st.integers(), min_size=1, max_size=10)
st.sets(st.text())
st.dictionaries(st.text(), st.integers())
st.tuples(st.text(), st.integers())
typescript
fc.array(fc.integer())
fc.array(fc.integer(), { minLength: 1, maxLength: 10 })
fc.set(fc.string())
fc.dictionary(fc.string(), fc.integer())
fc.tuple(fc.string(), fc.integer())

Dates:

python
st.datetimes()
st.dates(min_value=date(2020, 1, 1))
st.times()
typescript
fc.date()
fc.date({ min: new Date('2020-01-01') })

8. Best Practices

DO:

  • Test invariants, not specific outputs
  • Use meaningful property names
  • Start with simple properties
  • Let the library shrink failures
  • Test edge cases (empty, single item, max size)
  • Combine multiple properties
  • Use preconditions (assume in Hypothesis, fc.pre in fast-check)

DON'T:

  • Test implementation details
  • Use too complex properties
  • Ignore shrinking results
  • Forget to test edge cases
  • Make properties too similar to implementation
  • Use property-based tests for everything (unit tests still valuable)

9. Common Patterns

Metamorphic Testing:

python
@given(st.lists(st.integers()))
def test_sort_stability(items):
    # Adding an element and sorting should give same order for original elements
    with_extra = items + [max(items) + 1] if items else [0]
    sorted_original = sorted(items)
    sorted_with_extra = sorted(with_extra)

    # Original elements should appear in same relative order
    assert sorted_original == [x for x in sorted_with_extra if x in sorted_original]

Differential Testing:

typescript
// Test two implementations against each other
fc.assert(
  fc.property(fc.array(fc.integer()), (arr) => {
    const result1 = optimizedSort(arr);
    const result2 = naiveSort(arr);
    return JSON.stringify(result1) === JSON.stringify(result2);
  })
);

10. Integration with CI/CD

pytest.ini (Hypothesis):

ini
[pytest]
addopts =
    --hypothesis-show-statistics
    --hypothesis-seed=0

[hypothesis]
max_examples = 200
deadline = None

package.json (fast-check):

json
{
  "scripts": {
    "test": "jest",
    "test:property": "jest --testNamePattern='property'",
    "test:verbose": "jest --verbose"
  }
}

GitHub Actions:

yaml
name: Property Tests

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Install dependencies
        run: npm ci
      - name: Run property tests
        run: npm run test:property

11. Debugging Failed Properties

Hypothesis:

python
from hypothesis import given, strategies as st, example

@given(st.integers())
@example(0)  # Add specific examples to always test
@example(-1)
@example(999999)
def test_with_examples(x):
    assert process(x) >= 0

# Run with verbose output
# pytest --hypothesis-verbosity=verbose test_file.py

fast-check:

typescript
fc.assert(
  fc.property(fc.integer(), (x) => {
    // Use fc.pre for preconditions
    fc.pre(x !== 0);
    return 100 / x > 0;
  }),
  {
    seed: 1234567890,  // Reproduce exact failure
    path: "0:1:2",     // Replay specific path
    verbose: true
  }
);

12. Generate Test Report

Create a summary of property tests:

markdown
# Property-Based Test Report

## Coverage
- Functions tested: 15
- Properties verified: 42
- Test cases generated: 50,000+
- Edge cases found: 8

## Properties Tested

### sort_list
- ✅ Preserves length
- ✅ Creates ordered output
- ✅ Preserves all elements
- ✅ Handles empty lists
- ✅ Handles duplicates

### encode_decode
- ✅ Roundtrip property (decode(encode(x)) === x)
- ✅ Handles special characters
- ✅ Preserves data types

### merge_sorted_arrays
- ✅ Output is sorted
- ✅ Contains all elements
- ✅ Length equals sum of inputs

## Bugs Found
1. Division by zero in calculation (fixed)
2. Off-by-one error in array indexing (fixed)
3. Unicode handling issue in string processing (fixed)

## Recommendations
- Add property tests for user authentication flow
- Test database query builder invariants
- Add metamorphic tests for caching layer

Checklist

  • Property-based testing library installed
  • Function invariants identified
  • Basic properties implemented
  • Edge cases covered
  • Shrinking verified
  • CI/CD integration added
  • Documentation updated
  • Team trained on property-based testing

Recommended Agent Skills

Expand your agent's capabilities with these related and highly-rated skills.

Dexploarer/claudius-skills

smart-contract-generator

Generates Solidity smart contracts with security best practices (ERC-20, ERC-721, ERC-1155, custom). Use when user asks to "create smart contract", "solidity contract", "erc20 token", "nft contract", or "web3 contract".

5 2
Explore
Dexploarer/claudius-skills

threejs-scene-builder

Comprehensive Three.js and React Three Fiber skill for creating 3D scenes, characters, NPCs, procedural generation, animation retargeting, and interactive experiences. Use when user asks to "create Three.js scene", "setup React Three Fiber", "add 3D character", "create NPC AI", "procedural 3D generation", "retarget animation", "setup avatar system", or "create 3D game".

5 2
Explore
Dexploarer/claudius-skills

wcag-compliance-checker

Checks websites for WCAG 2.1 Level AA compliance, identifies accessibility violations, and provides remediation guidance. Use when user asks to "check accessibility", "wcag compliance", "a11y audit", "accessibility violations", or "screen reader testing".

5 2
Explore
Dexploarer/claudius-skills

kubernetes-manifest-generator

Generates Kubernetes manifests (Deployments, Services, Ingress, ConfigMaps, Secrets) with best practices for production workloads. Use when user asks to "create k8s manifest", "generate Kubernetes deployment", "setup k8s service", or "create Kubernetes resources".

5 2
Explore
Dexploarer/claudius-skills

terraform-module-builder

Generates reusable Terraform modules with best practices for AWS, Azure, GCP infrastructure as code. Use when user asks to "create Terraform module", "generate IaC module", "setup Terraform", or "create infrastructure module".

5 2
Explore
Dexploarer/claudius-skills

translation-key-extractor

Extracts hardcoded strings from code and converts them to translation keys for i18n. Use when user asks to "extract translations", "find hardcoded strings", "internationalize code", "setup i18n", or "create translation files".

5 2
Explore

Didn't find tool you were looking for?

Be as detailed as possible for better results