Agent skill
skill-quality-eval
Validate and audit quality of AI agent slash commands (.claude/commands/*.md) — checks YAML frontmatter, description length, jargon detection, directive language, and duplicate detection. Use when creating, editing, or auditing project commands and skills. Run proactively after adding new commands.
Install this agent skill to your Project
npx add-skill https://github.com/majiayu000/claude-skill-registry/tree/main/skills/other/other/skill-quality-eval
SKILL.md
Skill Quality Evaluator
Validates .claude/commands/*.md files for quality, triggering accuracy, and best practices from Phil Schmid's (Google DeepMind) eval harness methodology.
What it checks
- YAML frontmatter — every command must start with
---block containingdescription: - Description length — minimum 40 characters (short descriptions don't trigger well)
- Jargon detection — flags internal terms that confuse users (dogfood, --dangerously, etc.)
- Directive language — descriptions must contain action verbs (Run, Fix, Scan, Extract, etc.) because passive descriptions don't trigger Claude's skill routing
- Duplicate detection — similar commands must have distinct descriptions (first 30 chars normalized)
How to run
Execute the bundled script — it requires zero dependencies (pure Node.js):
node .claude/skills/skill-quality-eval/scripts/run-eval.js [commands-dir]
- Default
commands-dir:.claude/commands(relative to cwd) - Pass an absolute path to scan a different directory
The script outputs a structured report with PASS/FAIL per file, summary stats, and specific fix suggestions.
When a check fails
| Check | How to fix |
|---|---|
| Missing frontmatter | Add ---\ndescription: "..."\n--- at top of file |
| Description too short | Expand to 40+ chars — explain WHEN to use and WHAT it does |
| Jargon detected | Replace internal terms with user-facing language |
| No directive verb | Add action verbs: "Run...", "Fix...", "Scan...", "Use when..." |
| Duplicate description | Make the first 30 chars unique between similar commands |
Customization
Edit scripts/run-eval.js to add:
- Custom jargon patterns (line ~15,
JARGON_PATTERNSarray) - Additional directive verbs (line ~30,
DIRECTIVE_PATTERNSarray) - Differentiation pairs (line ~50,
DIFF_PAIRSarray)
Integration with CI
The script exits with code 1 if any check fails, making it suitable for pre-commit hooks or CI pipelines:
node .claude/skills/skill-quality-eval/scripts/run-eval.js || echo "Fix command quality issues!"
Background
Based on Phil Schmid's (Google DeepMind) eval harness methodology. See references/phil-schmid-eval-harness.md for the full framework and best practices.
Recommended Agent Skills
Expand your agent's capabilities with these related and highly-rated skills.
majiayu000/claude-skill-registry
agent-ops-spec
Manage specification documents in .agent/specs/. Use when user provides requirements, acceptance criteria, or feature descriptions that need to be tracked and validated against implementation.
majiayu000/claude-skill-registry
agent-ops-state
Maintain .agent state files. Use at session start, after meaningful steps, and before concluding: read/update constitution/memory/focus/issues/baseline consistently.
majiayu000/claude-skill-registry
agent-ops-spec
Manage specification documents in .agent/specs/. Use when user provides requirements, acceptance criteria, or feature descriptions that need to be tracked and validated against implementation.
majiayu000/claude-skill-registry
agent-ops-testing
Test strategy, execution, and coverage analysis. Use when designing tests, running test suites, or analyzing test results beyond baseline checks.
majiayu000/claude-skill-registry
agent-ops-testing
Test strategy, execution, and coverage analysis. Use when designing tests, running test suites, or analyzing test results beyond baseline checks.
majiayu000/claude-skill-registry
agent-ops-state
Maintain .agent state files. Use at session start, after meaningful steps, and before concluding: read/update constitution/memory/focus/issues/baseline consistently.
Didn't find tool you were looking for?