Agent skill

documentation-audit

This skill should be used when verifying documentation claims against codebase reality. Triggers on "audit docs", "verify documentation", "check docs", "docs accurate", "documentation drift", "before release", "after refactor", "docs don't match". Uses two-pass extraction with pattern expansion for comprehensive detection.

View SKILL.md on GitHub Repository
Stars 36
Forks 1

Install this agent skill to your Project

npx add-skill https://github.com/2389-research/claude-plugins/tree/main/documentation-audit/skills

SKILL.md

Documentation Audit

Systematically verify claims in documentation against the actual codebase using a two-pass approach.

Overview

Core principle: Low recall is worse than false positives—missed claims stay invisible.

Two-pass process:

  1. Pass 1: Extract and verify claims directly from docs
  2. Pass 2A: Expand patterns from false claims to find similar issues
  3. Pass 2B: Compare codebase inventory vs documented items (gap detection)

Quick Start

  1. Identify target docs (user-facing only, skip plans/, audits/)
  2. Note current git commit for report header
  3. Run Pass 1 extraction using parallel agents (one per doc)
  4. Analyze false claims for patterns
  5. Run Pass 2 expansion searches
  6. Generate docs/audits/AUDIT_REPORT_YYYY-MM-DD.md

Claim Types

Type Example Verification
file_ref scripts/foo.py File exists?
config_default "defaults to 'AI Radio'" Check schema/code
env_var STATION_NAME In .env.example + code?
cli_command --normalize flag Script supports it?
behavior "runs every 2 minutes" Check timers/code

Verification confidence:

  • Tier 1 (auto): file_ref, config_default, env_var, cli_command
  • Tier 2 (semi-auto): symbol_ref, version_req
  • Tier 3 (human review): behavior, constraint

Pass 2 Pattern Expansion

After Pass 1, analyze false claims and search for similar patterns:

Dead script found: diagnose_track_selection.py
  → Search: all script references → Found 8 more dead scripts

Wrong interval: "every 10 seconds"
  → Search: "every \d+ (seconds?|minutes?)" → Found 3 more

Wrong service name: ai-radio-break-gen.service
  → Search: service/timer names → Found naming inconsistencies

Common patterns to always check:

  • Dead scripts: scripts/*.py references
  • Timer intervals: every \d+ (seconds?|minutes?)
  • Service names: ai-radio-*.service, *.timer
  • Config vars: RADIO_* environment variables
  • CLI flags: --flag patterns in bash blocks

Output Format

Generate docs/audits/AUDIT_REPORT_YYYY-MM-DD.md:

markdown
# Documentation Audit Report
Generated: YYYY-MM-DD | Commit: abc123

## Executive Summary
| Metric | Count |
|--------|-------|
| Documents scanned | 12 |
| Claims verified | ~180 |
| Verified TRUE | ~145 (81%) |
| **Verified FALSE** | **31 (17%)** |

## False Claims Requiring Fixes
### CONFIGURATION.md
| Line | Claim | Reality | Fix |
|------|-------|---------|-----|
| 135 | `claude-sonnet-4-5` | Actual: `claude-3-5-sonnet-latest` | Update |

## Pattern Summary
| Pattern | Count | Root Cause |
|---------|-------|------------|
| Dead scripts | 9 | Scripts deleted, docs not updated |

## Human Review Queue
- [ ] Line 436: behavior claim needs verification

Detailed References

For execution checklist and anti-patterns: checklist.md For claim extraction patterns: extraction-patterns.md

Recommended Agent Skills

Expand your agent's capabilities with these related and highly-rated skills.

2389-research/claude-plugins

css-development

This skill should be used when working with CSS, creating components, styling elements, refactoring styles, or reviewing CSS code. Triggers on "CSS", "styles", "Tailwind", "dark mode", "component styling", "semantic class", "@apply", "stylesheet". Routes to specialized sub-skills for creation, validation, or refactoring.

36 1
Explore
2389-research/claude-plugins

css-development:create-component

This skill should be used when creating new styled components or adding new CSS classes. Triggers on "create component", "new button", "new card", "add styles", "style component", "build UI element". Guides semantic naming, Tailwind composition, dark mode support, and test coverage.

36 1
Explore
2389-research/claude-plugins

css-development:refactor

This skill should be used when refactoring existing CSS from inline styles or utility classes to semantic patterns. Triggers on "refactor CSS", "extract styles", "consolidate CSS", "convert inline", "clean up styles", "migrate to semantic". Transforms to semantic classes with dark mode and tests.

36 1
Explore
2389-research/claude-plugins

css-development:validate

This skill should be used when reviewing or auditing existing CSS code for consistency with established patterns. Triggers on "review CSS", "audit styles", "check CSS", "validate stylesheet", "CSS review". Checks semantic naming, dark mode coverage, Tailwind usage, and test coverage.

36 1
Explore
2389-research/claude-plugins

ceo-personal-os

This skill should be used when building a personal productivity or operating system for a CEO, founder, or executive. Triggers on "personal OS", "annual review", "life planning", "goal setting system", "Bill Campbell", "Trillion Dollar Coach", "startup failure patterns", "Good to Great", "Level 5 Leadership", "Buy Back Your Time", "E-Myth", "Customer Development", "Steve Blank", "Small Is Beautiful", "Schumacher", "human-scale", "subsidiarity", "Buddhist economics", "permanence".

36 1
Explore
2389-research/claude-plugins

gtm-partner

Strategic go-to-market partner that recommends channels, validates strategy with the user, and generates only the assets that matter. Use when a user has a validated business idea and needs tailored GTM strategy, not generic marketing assets.

36 1
Explore

Didn't find tool you were looking for?

Be as detailed as possible for better results