Agent skill
faion-testing-skill
Install this agent skill to your Project
npx add-skill https://github.com/majiayu000/claude-skill-registry/tree/main/skills/other/faion-testing-skill
SKILL.md
Testing Skill
Communication: User's language. Docs/code: English.
Purpose
Provides comprehensive testing patterns across languages and frameworks. This technical skill (Layer 3) is used by faion-test-agent to implement testing strategies.
3-Layer Architecture
Layer 1: Domain Skills ─ orchestrators
↓ call
Layer 2: Agents (faion-test-agent) ─ executors
↓ use
Layer 3: Technical Skills (this) ─ tools
Section 1: pytest (Python)
Overview
pytest is the most popular Python testing framework. Powerful fixtures, parametrization, and plugin ecosystem.
Installation
pip install pytest pytest-cov pytest-mock pytest-asyncio pytest-xdist
Project Structure
project/
├── src/
│ └── myapp/
│ ├── __init__.py
│ ├── services.py
│ └── models.py
├── tests/
│ ├── __init__.py
│ ├── conftest.py # Shared fixtures
│ ├── unit/
│ │ ├── __init__.py
│ │ ├── test_services.py
│ │ └── test_models.py
│ ├── integration/
│ │ └── test_api.py
│ └── e2e/
│ └── test_workflows.py
└── pyproject.toml
Configuration (pyproject.toml)
[tool.pytest.ini_options]
testpaths = ["tests"]
python_files = ["test_*.py"]
python_classes = ["Test*"]
python_functions = ["test_*"]
addopts = [
"-v",
"--strict-markers",
"--tb=short",
"-ra",
]
markers = [
"slow: marks tests as slow",
"integration: marks integration tests",
"e2e: marks end-to-end tests",
]
filterwarnings = [
"error",
"ignore::DeprecationWarning",
]
asyncio_mode = "auto"
[tool.coverage.run]
branch = true
source = ["src"]
omit = ["tests/*", "*/__init__.py"]
[tool.coverage.report]
exclude_lines = [
"pragma: no cover",
"if TYPE_CHECKING:",
"raise NotImplementedError",
]
fail_under = 80
Test Naming Convention
# Pattern: test_{what}_{when}_{expected}
def test_user_create_with_valid_data_returns_user():
pass
def test_user_create_with_invalid_email_raises_validation_error():
pass
def test_order_total_with_discount_calculates_correctly():
pass
Basic Test Structure (Arrange-Act-Assert)
import pytest
from myapp.services import UserService
class TestUserService:
"""Test suite for UserService."""
def test_create_user_with_valid_data(self):
# Arrange
service = UserService()
user_data = {"name": "John", "email": "[email protected]"}
# Act
user = service.create(user_data)
# Assert
assert user.id is not None
assert user.name == "John"
assert user.email == "[email protected]"
def test_create_user_with_invalid_email_raises_error(self):
# Arrange
service = UserService()
user_data = {"name": "John", "email": "invalid"}
# Act & Assert
with pytest.raises(ValueError, match="Invalid email"):
service.create(user_data)
Fixtures
Basic Fixtures
# conftest.py
import pytest
from myapp.database import Database
from myapp.services import UserService
@pytest.fixture
def db():
"""Create test database connection."""
database = Database(":memory:")
database.create_tables()
yield database
database.close()
@pytest.fixture
def user_service(db):
"""Create UserService with test database."""
return UserService(db)
@pytest.fixture
def sample_user(user_service):
"""Create a sample user for testing."""
return user_service.create({
"name": "Test User",
"email": "[email protected]"
})
Fixture Scopes
@pytest.fixture(scope="function") # Default: new for each test
def per_test_fixture():
pass
@pytest.fixture(scope="class") # Once per test class
def per_class_fixture():
pass
@pytest.fixture(scope="module") # Once per test module
def per_module_fixture():
pass
@pytest.fixture(scope="session") # Once per test session
def per_session_fixture():
pass
Parameterized Fixtures
@pytest.fixture(params=["sqlite", "postgres", "mysql"])
def db_type(request):
"""Run tests with different databases."""
return request.param
@pytest.fixture
def db(db_type):
"""Create database based on type."""
if db_type == "sqlite":
return SQLiteDatabase()
elif db_type == "postgres":
return PostgresDatabase()
else:
return MySQLDatabase()
Factory Fixtures
@pytest.fixture
def user_factory(db):
"""Factory for creating test users."""
created_users = []
def _create_user(**kwargs):
defaults = {
"name": "Test User",
"email": f"user{len(created_users)}@example.com"
}
defaults.update(kwargs)
user = User(**defaults)
db.add(user)
created_users.append(user)
return user
yield _create_user
# Cleanup
for user in created_users:
db.delete(user)
Parametrization
Basic Parametrize
@pytest.mark.parametrize("input,expected", [
(1, 2),
(2, 4),
(3, 6),
(0, 0),
(-1, -2),
])
def test_double(input, expected):
assert double(input) == expected
Multiple Parameters
@pytest.mark.parametrize("x", [0, 1])
@pytest.mark.parametrize("y", [2, 3])
def test_combinations(x, y):
# Runs 4 times: (0,2), (0,3), (1,2), (1,3)
assert x + y >= 2
Parametrize with IDs
@pytest.mark.parametrize("email,valid", [
pytest.param("[email protected]", True, id="valid_email"),
pytest.param("invalid", False, id="no_at_symbol"),
pytest.param("@example.com", False, id="no_local_part"),
pytest.param("user@", False, id="no_domain"),
])
def test_email_validation(email, valid):
assert validate_email(email) == valid
Mocking
Using pytest-mock
def test_send_email_calls_smtp(mocker):
# Arrange
mock_smtp = mocker.patch("myapp.email.smtplib.SMTP")
service = EmailService()
# Act
service.send("[email protected]", "Hello", "Body")
# Assert
mock_smtp.return_value.sendmail.assert_called_once()
def test_external_api_call(mocker):
# Mock external API response
mock_response = mocker.Mock()
mock_response.json.return_value = {"status": "ok"}
mock_response.status_code = 200
mocker.patch("requests.get", return_value=mock_response)
result = fetch_data("https://api.example.com")
assert result["status"] == "ok"
Mocking Classes
def test_with_mock_class(mocker):
# Create mock class instance
mock_client = mocker.Mock(spec=DatabaseClient)
mock_client.query.return_value = [{"id": 1, "name": "Test"}]
# Inject mock
service = UserService(client=mock_client)
users = service.get_all()
assert len(users) == 1
mock_client.query.assert_called_with("SELECT * FROM users")
Side Effects
def test_retry_on_failure(mocker):
# Simulate failures then success
mock_api = mocker.patch("myapp.api.external_call")
mock_api.side_effect = [
ConnectionError("Network error"),
ConnectionError("Network error"),
{"result": "success"},
]
result = api_with_retry(max_retries=3)
assert result == {"result": "success"}
assert mock_api.call_count == 3
Markers
Built-in Markers
@pytest.mark.skip(reason="Not implemented yet")
def test_future_feature():
pass
@pytest.mark.skipif(sys.version_info < (3, 10), reason="Requires Python 3.10+")
def test_new_syntax():
pass
@pytest.mark.xfail(reason="Known bug, fix in progress")
def test_known_issue():
pass
Custom Markers
# In conftest.py
def pytest_configure(config):
config.addinivalue_line("markers", "slow: marks tests as slow")
config.addinivalue_line("markers", "integration: integration tests")
# In tests
@pytest.mark.slow
def test_heavy_computation():
pass
@pytest.mark.integration
def test_database_connection():
pass
# Run specific markers
# pytest -m "slow"
# pytest -m "not integration"
Async Testing
import pytest
import asyncio
@pytest.mark.asyncio
async def test_async_function():
result = await async_fetch_data()
assert result is not None
@pytest.mark.asyncio
async def test_concurrent_operations():
results = await asyncio.gather(
async_operation_1(),
async_operation_2(),
async_operation_3(),
)
assert len(results) == 3
# Async fixture
@pytest.fixture
async def async_db():
db = await AsyncDatabase.connect()
yield db
await db.close()
Running pytest
# Run all tests
pytest
# Verbose output
pytest -v
# Run specific file
pytest tests/test_services.py
# Run specific test
pytest tests/test_services.py::TestUserService::test_create_user
# Run tests matching pattern
pytest -k "create or delete"
# Run tests with marker
pytest -m integration
# Parallel execution (pytest-xdist)
pytest -n auto
# Show coverage
pytest --cov=src --cov-report=html
# Stop on first failure
pytest -x
# Run last failed tests
pytest --lf
# Run failed tests first
pytest --ff
Section 2: Jest/Vitest (JavaScript/TypeScript)
Overview
Jest is the most popular JavaScript testing framework. Vitest is a newer, faster alternative compatible with Jest API.
Installation
# Jest
npm install --save-dev jest @types/jest ts-jest
# Vitest
npm install --save-dev vitest @vitest/coverage-v8
Project Structure
project/
├── src/
│ ├── services/
│ │ └── userService.ts
│ └── utils/
│ └── helpers.ts
├── tests/
│ ├── setup.ts
│ ├── services/
│ │ └── userService.test.ts
│ └── utils/
│ └── helpers.test.ts
├── jest.config.js # or vitest.config.ts
└── package.json
Jest Configuration (jest.config.js)
/** @type {import('jest').Config} */
const config = {
preset: 'ts-jest',
testEnvironment: 'node',
roots: ['<rootDir>/tests'],
testMatch: ['**/*.test.ts', '**/*.spec.ts'],
moduleNameMapper: {
'^@/(.*)$': '<rootDir>/src/$1',
},
setupFilesAfterEnv: ['<rootDir>/tests/setup.ts'],
collectCoverageFrom: [
'src/**/*.{ts,tsx}',
'!src/**/*.d.ts',
'!src/index.ts',
],
coverageThreshold: {
global: {
branches: 80,
functions: 80,
lines: 80,
statements: 80,
},
},
clearMocks: true,
testTimeout: 10000,
};
module.exports = config;
Vitest Configuration (vitest.config.ts)
import { defineConfig } from 'vitest/config';
import path from 'path';
export default defineConfig({
test: {
globals: true,
environment: 'node',
include: ['tests/**/*.{test,spec}.{ts,tsx}'],
setupFiles: ['./tests/setup.ts'],
coverage: {
provider: 'v8',
reporter: ['text', 'html', 'lcov'],
include: ['src/**/*.ts'],
exclude: ['src/**/*.d.ts', 'src/index.ts'],
thresholds: {
branches: 80,
functions: 80,
lines: 80,
statements: 80,
},
},
testTimeout: 10000,
},
resolve: {
alias: {
'@': path.resolve(__dirname, './src'),
},
},
});
Basic Test Structure
// userService.test.ts
import { describe, it, expect, beforeEach, afterEach } from 'vitest'; // or jest
import { UserService } from '@/services/userService';
describe('UserService', () => {
let service: UserService;
beforeEach(() => {
service = new UserService();
});
afterEach(() => {
// Cleanup
});
describe('createUser', () => {
it('should create user with valid data', () => {
// Arrange
const userData = { name: 'John', email: '[email protected]' };
// Act
const user = service.createUser(userData);
// Assert
expect(user).toBeDefined();
expect(user.id).toBeDefined();
expect(user.name).toBe('John');
expect(user.email).toBe('[email protected]');
});
it('should throw error with invalid email', () => {
// Arrange
const userData = { name: 'John', email: 'invalid' };
// Act & Assert
expect(() => service.createUser(userData)).toThrow('Invalid email');
});
});
});
Matchers
describe('Matchers', () => {
// Equality
it('equality matchers', () => {
expect(2 + 2).toBe(4); // Strict equality
expect({ a: 1 }).toEqual({ a: 1 }); // Deep equality
expect({ a: 1, b: 2 }).toMatchObject({ a: 1 }); // Partial match
});
// Truthiness
it('truthiness matchers', () => {
expect(null).toBeNull();
expect(undefined).toBeUndefined();
expect('hello').toBeDefined();
expect(true).toBeTruthy();
expect(false).toBeFalsy();
});
// Numbers
it('number matchers', () => {
expect(4).toBeGreaterThan(3);
expect(4).toBeGreaterThanOrEqual(4);
expect(4).toBeLessThan(5);
expect(0.1 + 0.2).toBeCloseTo(0.3);
});
// Strings
it('string matchers', () => {
expect('hello world').toContain('world');
expect('hello').toMatch(/^hel/);
expect('hello').toHaveLength(5);
});
// Arrays
it('array matchers', () => {
expect([1, 2, 3]).toContain(2);
expect([1, 2, 3]).toHaveLength(3);
expect(['a', 'b', 'c']).toEqual(expect.arrayContaining(['a', 'c']));
});
// Objects
it('object matchers', () => {
expect({ name: 'John' }).toHaveProperty('name');
expect({ name: 'John' }).toHaveProperty('name', 'John');
expect({ a: 1, b: 2 }).toEqual(expect.objectContaining({ a: 1 }));
});
// Exceptions
it('exception matchers', () => {
expect(() => { throw new Error('fail'); }).toThrow();
expect(() => { throw new Error('fail'); }).toThrow('fail');
expect(() => { throw new Error('fail'); }).toThrow(Error);
});
});
Mocking
Function Mocks
import { vi, describe, it, expect, beforeEach } from 'vitest';
describe('Mocking', () => {
it('should mock function', () => {
const mockFn = vi.fn();
mockFn('arg1', 'arg2');
expect(mockFn).toHaveBeenCalled();
expect(mockFn).toHaveBeenCalledWith('arg1', 'arg2');
expect(mockFn).toHaveBeenCalledTimes(1);
});
it('should mock return value', () => {
const mockFn = vi.fn()
.mockReturnValue('default')
.mockReturnValueOnce('first')
.mockReturnValueOnce('second');
expect(mockFn()).toBe('first');
expect(mockFn()).toBe('second');
expect(mockFn()).toBe('default');
});
it('should mock implementation', () => {
const mockFn = vi.fn().mockImplementation((x: number) => x * 2);
expect(mockFn(5)).toBe(10);
});
});
Module Mocks
// Mock entire module
vi.mock('@/services/api', () => ({
fetchData: vi.fn().mockResolvedValue({ data: 'mocked' }),
}));
// Mock specific export
vi.mock('@/utils/helpers', async (importOriginal) => {
const actual = await importOriginal();
return {
...actual,
formatDate: vi.fn().mockReturnValue('2024-01-01'),
};
});
// Spy on module
import * as helpers from '@/utils/helpers';
it('should spy on module', () => {
const spy = vi.spyOn(helpers, 'formatDate');
helpers.formatDate(new Date());
expect(spy).toHaveBeenCalled();
});
Mocking HTTP Requests
import { vi } from 'vitest';
// Mock fetch
global.fetch = vi.fn();
describe('API calls', () => {
beforeEach(() => {
vi.mocked(fetch).mockReset();
});
it('should fetch data', async () => {
vi.mocked(fetch).mockResolvedValueOnce({
ok: true,
json: () => Promise.resolve({ users: [] }),
} as Response);
const result = await fetchUsers();
expect(fetch).toHaveBeenCalledWith('/api/users');
expect(result).toEqual({ users: [] });
});
it('should handle error', async () => {
vi.mocked(fetch).mockRejectedValueOnce(new Error('Network error'));
await expect(fetchUsers()).rejects.toThrow('Network error');
});
});
Async Testing
describe('Async', () => {
// Async/await
it('should resolve async', async () => {
const result = await asyncFetchData();
expect(result).toBeDefined();
});
// Promises
it('should resolve promise', () => {
return fetchData().then(result => {
expect(result).toBeDefined();
});
});
// Callbacks
it('should call callback', (done) => {
fetchDataCallback((error, result) => {
expect(error).toBeNull();
expect(result).toBeDefined();
done();
});
});
// Timeouts
it('should handle timers', () => {
vi.useFakeTimers();
const callback = vi.fn();
setTimeout(callback, 1000);
vi.advanceTimersByTime(1000);
expect(callback).toHaveBeenCalled();
vi.useRealTimers();
});
});
Snapshot Testing
describe('Snapshots', () => {
it('should match snapshot', () => {
const user = { id: 1, name: 'John', email: '[email protected]' };
expect(user).toMatchSnapshot();
});
it('should match inline snapshot', () => {
const result = formatUser({ name: 'John' });
expect(result).toMatchInlineSnapshot(`"Hello, John!"`);
});
});
Running Tests
# Jest
npx jest
npx jest --watch
npx jest --coverage
npx jest path/to/test.ts
npx jest -t "test name pattern"
# Vitest
npx vitest
npx vitest run
npx vitest --coverage
npx vitest path/to/test.ts
npx vitest -t "test name pattern"
Section 3: Go Testing
Overview
Go has built-in testing support via the testing package. No external dependencies required.
Project Structure
project/
├── pkg/
│ └── user/
│ ├── user.go
│ ├── user_test.go
│ └── testdata/
│ └── fixtures.json
├── internal/
│ └── service/
│ ├── service.go
│ └── service_test.go
├── go.mod
└── go.sum
Basic Test Structure
// user_test.go
package user
import (
"testing"
)
func TestCreateUser(t *testing.T) {
// Arrange
service := NewUserService()
input := UserInput{Name: "John", Email: "[email protected]"}
// Act
user, err := service.Create(input)
// Assert
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if user.Name != "John" {
t.Errorf("expected name 'John', got '%s'", user.Name)
}
if user.Email != "[email protected]" {
t.Errorf("expected email '[email protected]', got '%s'", user.Email)
}
}
Table-Driven Tests
func TestValidateEmail(t *testing.T) {
tests := []struct {
name string
email string
wantErr bool
}{
{
name: "valid email",
email: "[email protected]",
wantErr: false,
},
{
name: "missing at symbol",
email: "invalid",
wantErr: true,
},
{
name: "missing domain",
email: "user@",
wantErr: true,
},
{
name: "empty string",
email: "",
wantErr: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
err := ValidateEmail(tt.email)
if (err != nil) != tt.wantErr {
t.Errorf("ValidateEmail(%q) error = %v, wantErr %v",
tt.email, err, tt.wantErr)
}
})
}
}
Subtests
func TestUserService(t *testing.T) {
service := NewUserService()
t.Run("Create", func(t *testing.T) {
t.Run("with valid data", func(t *testing.T) {
user, err := service.Create(validInput)
if err != nil {
t.Fatal(err)
}
if user.ID == 0 {
t.Error("expected non-zero ID")
}
})
t.Run("with invalid email", func(t *testing.T) {
_, err := service.Create(invalidInput)
if err == nil {
t.Error("expected error for invalid email")
}
})
})
t.Run("Update", func(t *testing.T) {
// Update tests
})
}
Test Helpers
// testutil/helpers.go
package testutil
import "testing"
// AssertNoError fails the test if err is not nil
func AssertNoError(t *testing.T, err error) {
t.Helper()
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
}
// AssertError fails the test if err is nil
func AssertError(t *testing.T, err error) {
t.Helper()
if err == nil {
t.Fatal("expected error, got nil")
}
}
// AssertEqual fails if got != want
func AssertEqual[T comparable](t *testing.T, got, want T) {
t.Helper()
if got != want {
t.Errorf("got %v, want %v", got, want)
}
}
Mocking with Interfaces
// Define interface
type UserRepository interface {
Create(user User) (User, error)
FindByID(id int) (User, error)
}
// Mock implementation
type MockUserRepository struct {
CreateFunc func(user User) (User, error)
FindByIDFunc func(id int) (User, error)
}
func (m *MockUserRepository) Create(user User) (User, error) {
if m.CreateFunc != nil {
return m.CreateFunc(user)
}
return User{}, nil
}
func (m *MockUserRepository) FindByID(id int) (User, error) {
if m.FindByIDFunc != nil {
return m.FindByIDFunc(id)
}
return User{}, nil
}
// Test with mock
func TestUserService_Create(t *testing.T) {
mockRepo := &MockUserRepository{
CreateFunc: func(user User) (User, error) {
user.ID = 1
return user, nil
},
}
service := NewUserService(mockRepo)
user, err := service.Create(UserInput{Name: "John"})
if err != nil {
t.Fatal(err)
}
if user.ID != 1 {
t.Errorf("expected ID 1, got %d", user.ID)
}
}
Setup and Teardown
func TestMain(m *testing.M) {
// Global setup
setup()
// Run tests
code := m.Run()
// Global teardown
teardown()
os.Exit(code)
}
func setup() {
// Initialize test database, etc.
}
func teardown() {
// Cleanup
}
Benchmarks
func BenchmarkProcessData(b *testing.B) {
data := generateTestData(1000)
b.ResetTimer()
for i := 0; i < b.N; i++ {
ProcessData(data)
}
}
func BenchmarkProcessData_Parallel(b *testing.B) {
data := generateTestData(1000)
b.RunParallel(func(pb *testing.PB) {
for pb.Next() {
ProcessData(data)
}
})
}
Fuzzing (Go 1.18+)
func FuzzParseEmail(f *testing.F) {
// Add seed corpus
f.Add("[email protected]")
f.Add("invalid")
f.Add("")
f.Fuzz(func(t *testing.T, email string) {
_, err := ParseEmail(email)
// Just verify it doesn't panic
_ = err
})
}
Running Go Tests
# Run all tests
go test ./...
# Verbose output
go test -v ./...
# Run specific package
go test ./pkg/user
# Run specific test
go test -run TestCreateUser ./pkg/user
# Run subtests
go test -run TestUserService/Create/with_valid_data ./pkg/user
# With coverage
go test -cover ./...
go test -coverprofile=coverage.out ./...
go tool cover -html=coverage.out
# Run benchmarks
go test -bench=. ./...
go test -bench=BenchmarkProcessData -benchmem ./...
# Race detection
go test -race ./...
# Timeout
go test -timeout 30s ./...
# Run tests in parallel
go test -parallel 4 ./...
Section 4: E2E Testing
Playwright (Recommended)
Installation
npm install --save-dev @playwright/test
npx playwright install
Configuration (playwright.config.ts)
import { defineConfig, devices } from '@playwright/test';
export default defineConfig({
testDir: './e2e',
fullyParallel: true,
forbidOnly: !!process.env.CI,
retries: process.env.CI ? 2 : 0,
workers: process.env.CI ? 1 : undefined,
reporter: [
['html'],
['junit', { outputFile: 'results.xml' }],
],
use: {
baseURL: 'http://localhost:3000',
trace: 'on-first-retry',
screenshot: 'only-on-failure',
video: 'on-first-retry',
},
projects: [
{
name: 'chromium',
use: { ...devices['Desktop Chrome'] },
},
{
name: 'firefox',
use: { ...devices['Desktop Firefox'] },
},
{
name: 'webkit',
use: { ...devices['Desktop Safari'] },
},
{
name: 'Mobile Chrome',
use: { ...devices['Pixel 5'] },
},
],
webServer: {
command: 'npm run start',
url: 'http://localhost:3000',
reuseExistingServer: !process.env.CI,
},
});
Basic Test Structure
import { test, expect } from '@playwright/test';
test.describe('User Authentication', () => {
test.beforeEach(async ({ page }) => {
await page.goto('/');
});
test('should login with valid credentials', async ({ page }) => {
// Navigate to login
await page.click('[data-testid="login-button"]');
// Fill form
await page.fill('[data-testid="email-input"]', '[email protected]');
await page.fill('[data-testid="password-input"]', 'password123');
// Submit
await page.click('[data-testid="submit-button"]');
// Assert
await expect(page).toHaveURL('/dashboard');
await expect(page.locator('[data-testid="welcome-message"]'))
.toContainText('Welcome');
});
test('should show error for invalid credentials', async ({ page }) => {
await page.click('[data-testid="login-button"]');
await page.fill('[data-testid="email-input"]', '[email protected]');
await page.fill('[data-testid="password-input"]', 'wrongpassword');
await page.click('[data-testid="submit-button"]');
await expect(page.locator('[data-testid="error-message"]'))
.toBeVisible();
});
});
Page Object Pattern
// pages/LoginPage.ts
import { Page, Locator } from '@playwright/test';
export class LoginPage {
readonly page: Page;
readonly emailInput: Locator;
readonly passwordInput: Locator;
readonly submitButton: Locator;
readonly errorMessage: Locator;
constructor(page: Page) {
this.page = page;
this.emailInput = page.locator('[data-testid="email-input"]');
this.passwordInput = page.locator('[data-testid="password-input"]');
this.submitButton = page.locator('[data-testid="submit-button"]');
this.errorMessage = page.locator('[data-testid="error-message"]');
}
async goto() {
await this.page.goto('/login');
}
async login(email: string, password: string) {
await this.emailInput.fill(email);
await this.passwordInput.fill(password);
await this.submitButton.click();
}
}
// Using in tests
import { test, expect } from '@playwright/test';
import { LoginPage } from './pages/LoginPage';
test('should login', async ({ page }) => {
const loginPage = new LoginPage(page);
await loginPage.goto();
await loginPage.login('[email protected]', 'password');
await expect(page).toHaveURL('/dashboard');
});
Fixtures
// fixtures.ts
import { test as base } from '@playwright/test';
import { LoginPage } from './pages/LoginPage';
import { DashboardPage } from './pages/DashboardPage';
type Fixtures = {
loginPage: LoginPage;
dashboardPage: DashboardPage;
authenticatedPage: Page;
};
export const test = base.extend<Fixtures>({
loginPage: async ({ page }, use) => {
await use(new LoginPage(page));
},
dashboardPage: async ({ page }, use) => {
await use(new DashboardPage(page));
},
authenticatedPage: async ({ page }, use) => {
// Login before test
const loginPage = new LoginPage(page);
await loginPage.goto();
await loginPage.login('[email protected]', 'password');
await page.waitForURL('/dashboard');
await use(page);
},
});
export { expect } from '@playwright/test';
API Testing with Playwright
import { test, expect } from '@playwright/test';
test.describe('API Tests', () => {
test('should create user via API', async ({ request }) => {
const response = await request.post('/api/users', {
data: {
name: 'John',
email: '[email protected]',
},
});
expect(response.status()).toBe(201);
const user = await response.json();
expect(user.id).toBeDefined();
expect(user.name).toBe('John');
});
test('should authenticate via API', async ({ request }) => {
const response = await request.post('/api/auth/login', {
data: {
email: '[email protected]',
password: 'password',
},
});
expect(response.status()).toBe(200);
const { token } = await response.json();
// Use token for subsequent requests
const profileResponse = await request.get('/api/me', {
headers: {
Authorization: `Bearer ${token}`,
},
});
expect(profileResponse.status()).toBe(200);
});
});
Running Playwright
# Run all tests
npx playwright test
# Run in headed mode
npx playwright test --headed
# Run specific file
npx playwright test e2e/auth.spec.ts
# Run specific project
npx playwright test --project=chromium
# Debug mode
npx playwright test --debug
# UI mode
npx playwright test --ui
# Generate tests
npx playwright codegen http://localhost:3000
# Show report
npx playwright show-report
Cypress (Alternative)
Installation
npm install --save-dev cypress
Configuration (cypress.config.ts)
import { defineConfig } from 'cypress';
export default defineConfig({
e2e: {
baseUrl: 'http://localhost:3000',
supportFile: 'cypress/support/e2e.ts',
specPattern: 'cypress/e2e/**/*.cy.{js,ts}',
viewportWidth: 1280,
viewportHeight: 720,
video: false,
screenshotOnRunFailure: true,
},
});
Basic Test
// cypress/e2e/auth.cy.ts
describe('Authentication', () => {
beforeEach(() => {
cy.visit('/');
});
it('should login successfully', () => {
cy.get('[data-testid="login-button"]').click();
cy.get('[data-testid="email-input"]').type('[email protected]');
cy.get('[data-testid="password-input"]').type('password');
cy.get('[data-testid="submit-button"]').click();
cy.url().should('include', '/dashboard');
cy.get('[data-testid="welcome-message"]').should('contain', 'Welcome');
});
});
Custom Commands
// cypress/support/commands.ts
Cypress.Commands.add('login', (email: string, password: string) => {
cy.visit('/login');
cy.get('[data-testid="email-input"]').type(email);
cy.get('[data-testid="password-input"]').type(password);
cy.get('[data-testid="submit-button"]').click();
cy.url().should('include', '/dashboard');
});
// Usage
cy.login('[email protected]', 'password');
Section 5: Test Structure Patterns
Arrange-Act-Assert (AAA)
def test_user_creation():
# Arrange - Set up test data and conditions
service = UserService()
user_data = {"name": "John", "email": "[email protected]"}
# Act - Perform the action being tested
user = service.create(user_data)
# Assert - Verify the expected outcome
assert user.id is not None
assert user.name == "John"
Given-When-Then (BDD)
def test_user_login():
# Given - A registered user exists
user = create_user("[email protected]", "password123")
# When - The user logs in with correct credentials
result = auth_service.login("[email protected]", "password123")
# Then - A valid session token is returned
assert result.token is not None
assert result.user_id == user.id
Test Isolation Principles
class TestUserService:
"""
Test Isolation Rules:
1. Each test is independent
2. Tests don't share state
3. Tests can run in any order
4. Tests clean up after themselves
"""
@pytest.fixture(autouse=True)
def setup(self, db):
"""Fresh database for each test."""
db.clear_all()
yield
db.clear_all()
def test_create_user(self, user_service):
# This test doesn't depend on other tests
user = user_service.create({"name": "John"})
assert user.id == 1 # First user in fresh db
def test_list_users(self, user_service):
# This test also starts with empty database
users = user_service.list_all()
assert len(users) == 0 # Fresh db
Test Pyramid
/\
/ \
/ E2E \ <- Few, slow, expensive
/------\
/ \
/ Integration \ <- Some, medium speed
/--------------\
/ \
/ Unit Tests \ <- Many, fast, cheap
/--------------------\
Recommended ratio: 70% Unit, 20% Integration, 10% E2E
Section 6: Mocking and Fixtures
When to Mock
| Mock | Don't Mock |
|---|---|
| External APIs | Your own code logic |
| Database in unit tests | Simple utilities |
| File system | Pure functions |
| Network calls | Domain logic |
| Time/randomness |
Fixture Strategies
Builder Pattern
class UserBuilder:
def __init__(self):
self._name = "Default User"
self._email = "[email protected]"
self._role = "user"
def with_name(self, name: str) -> "UserBuilder":
self._name = name
return self
def with_email(self, email: str) -> "UserBuilder":
self._email = email
return self
def as_admin(self) -> "UserBuilder":
self._role = "admin"
return self
def build(self) -> User:
return User(
name=self._name,
email=self._email,
role=self._role
)
# Usage
user = UserBuilder().with_name("John").as_admin().build()
Factory Pattern
import factory
from factory.faker import Faker
class UserFactory(factory.Factory):
class Meta:
model = User
name = Faker("name")
email = Faker("email")
created_at = factory.LazyFunction(datetime.now)
# Usage
user = UserFactory() # Random user
user = UserFactory(name="John") # Override specific field
users = UserFactory.create_batch(10) # Create many
Section 7: Coverage Reporting
Python (pytest-cov)
# Generate coverage report
pytest --cov=src --cov-report=html --cov-report=term
# Enforce minimum coverage
pytest --cov=src --cov-fail-under=80
# Coverage for specific paths
pytest --cov=src/services --cov=src/models tests/
JavaScript (Jest/Vitest)
# Jest
npx jest --coverage
# Vitest
npx vitest run --coverage
Go
# Generate coverage
go test -coverprofile=coverage.out ./...
# View in terminal
go tool cover -func=coverage.out
# Generate HTML report
go tool cover -html=coverage.out -o coverage.html
Coverage Best Practices
- Set realistic thresholds - Start at 70-80%, not 100%
- Focus on critical paths - Business logic > boilerplate
- Exclude generated code - Don't test auto-generated files
- Branch coverage matters - Not just line coverage
- Don't game metrics - Tests should verify behavior, not just execute code
Section 8: TDD Workflow
Red-Green-Refactor Cycle
RED: Write failing test
↓
GREEN: Write minimum code to pass
↓
REFACTOR: Improve code quality
↓
(repeat)
Example TDD Session
# Step 1: RED - Write failing test
def test_calculate_discount():
calculator = PriceCalculator()
result = calculator.calculate_discount(100, 10)
assert result == 90 # FAILS - class doesn't exist
# Step 2: GREEN - Minimal implementation
class PriceCalculator:
def calculate_discount(self, price: float, percent: float) -> float:
return price - (price * percent / 100)
# Step 3: REFACTOR - Improve
class PriceCalculator:
def calculate_discount(self, price: float, percent: float) -> float:
if percent < 0 or percent > 100:
raise ValueError("Discount must be between 0 and 100")
return round(price * (1 - percent / 100), 2)
# Step 4: Add more tests for edge cases
@pytest.mark.parametrize("price,discount,expected", [
(100, 10, 90),
(100, 0, 100),
(100, 100, 0),
(49.99, 20, 39.99),
])
def test_calculate_discount_cases(price, discount, expected):
calculator = PriceCalculator()
assert calculator.calculate_discount(price, discount) == expected
TDD Benefits
- Design feedback - Tests expose design problems early
- Documentation - Tests document expected behavior
- Confidence - Safe refactoring with test coverage
- Focus - Work on one thing at a time
Section 9: CI Integration
GitHub Actions Example
# .github/workflows/test.yml
name: Tests
on:
push:
branches: [main]
pull_request:
branches: [main]
jobs:
test-python:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: '3.11'
- name: Install dependencies
run: |
pip install -r requirements.txt
pip install -r requirements-dev.txt
- name: Run tests
run: pytest --cov=src --cov-report=xml
- name: Upload coverage
uses: codecov/codecov-action@v4
test-javascript:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: '20'
- name: Install dependencies
run: npm ci
- name: Run tests
run: npm test -- --coverage
test-go:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: '1.22'
- name: Run tests
run: go test -race -coverprofile=coverage.out ./...
- name: Upload coverage
uses: codecov/codecov-action@v4
e2e:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: '20'
- name: Install dependencies
run: npm ci
- name: Install Playwright
run: npx playwright install --with-deps
- name: Run E2E tests
run: npx playwright test
- uses: actions/upload-artifact@v4
if: failure()
with:
name: playwright-report
path: playwright-report/
Quick Reference
Commands
| Framework | Run All | Run Specific | Coverage |
|---|---|---|---|
| pytest | pytest |
pytest tests/test_x.py -k name |
pytest --cov=src |
| Jest | npx jest |
npx jest -t "name" |
npx jest --coverage |
| Vitest | npx vitest |
npx vitest -t "name" |
npx vitest --coverage |
| Go | go test ./... |
go test -run TestName |
go test -cover |
| Playwright | npx playwright test |
npx playwright test file.spec.ts |
N/A |
Test Naming
| Style | Example |
|---|---|
| Descriptive | test_user_create_with_valid_data_returns_user |
| BDD | should_return_user_when_data_is_valid |
| Simple | test_create_user_success |
Testing Skill v1.0 - 2026-01-18 Technical Skill (Layer 3) Used by: faion-test-agent
Recommended Agent Skills
Expand your agent's capabilities with these related and highly-rated skills.
agent-ops-spec
Manage specification documents in .agent/specs/. Use when user provides requirements, acceptance criteria, or feature descriptions that need to be tracked and validated against implementation.
agent-ops-state
Maintain .agent state files. Use at session start, after meaningful steps, and before concluding: read/update constitution/memory/focus/issues/baseline consistently.
agent-ops-spec
Manage specification documents in .agent/specs/. Use when user provides requirements, acceptance criteria, or feature descriptions that need to be tracked and validated against implementation.
agent-ops-testing
Test strategy, execution, and coverage analysis. Use when designing tests, running test suites, or analyzing test results beyond baseline checks.
agent-ops-testing
Test strategy, execution, and coverage analysis. Use when designing tests, running test suites, or analyzing test results beyond baseline checks.
agent-ops-state
Maintain .agent state files. Use at session start, after meaningful steps, and before concluding: read/update constitution/memory/focus/issues/baseline consistently.
Didn't find tool you were looking for?