Agent skill

skill-eval

Evaluate all workspace-hub skills for structural validity, content quality, cross-reference integrity, and registry consistency. Runs 18 checks across critical, warning, and info severity levels with actionable fix suggestions.

View SKILL.md on GitHub Repository
Stars 163
Forks 31

Install this agent skill to your Project

npx add-skill https://github.com/majiayu000/claude-skill-registry/tree/main/skills/other/skill-eval

SKILL.md

Skill Eval

Quick Start

bash
# Full evaluation of all skills
uv run .claude/skills/development/skill-eval/scripts/eval-skills.py

# JSON output
uv run .claude/skills/development/skill-eval/scripts/eval-skills.py --format json

# Single skill
uv run .claude/skills/development/skill-eval/scripts/eval-skills.py --skill testing-tdd-london

# Only critical issues
uv run .claude/skills/development/skill-eval/scripts/eval-skills.py --severity critical

When to Use

  • Auditing skill quality after bulk creation or migration
  • Pre-release validation of the skills library
  • CI/CD quality gate for skill changes
  • Identifying broken cross-references after renaming skills
  • Checking compliance with v2 SKILL.md template format

Core Concepts

Evaluation Dimensions

The evaluator runs 18 checks organized into three severity levels:

Critical (blocks skill usage):

  • YAML frontmatter exists and parses
  • Required fields present: name, description

Warning (degrades quality):

  • Version follows semver, category matches directory
  • Required content sections present (Quick Start, When to Use, etc.)
  • Code blocks in key sections, no TODO/FIXME markers
  • Cross-references in related_skills resolve to real skills

Info (improvement opportunities):

  • Uses v2 template format, has optional sections (Metrics, etc.)
  • No duplicate skill names across the library

Report Output

Reports include:

  • Summary with pass/fail counts and percentages
  • Issues grouped by severity with counts
  • Per-category breakdown
  • Top 10 most common issues
  • Per-skill details with actionable fix suggestions

Usage Examples

Full Evaluation

bash
uv run .claude/skills/development/skill-eval/scripts/eval-skills.py

Output:

================================================================
  SKILL EVALUATION REPORT
  Generated: 2026-01-29T14:30:00+00:00
================================================================

SUMMARY
----------------------------------------
  Total skills evaluated:  230
  Passed (no critical):    187  (81.3%)
  Warnings only:           102  (44.3%)
  Critical failures:        43  (18.7%)

JSON for CI/CD

bash
uv run .claude/skills/development/skill-eval/scripts/eval-skills.py \
  --format json --severity critical \
  --output reports/skill-eval.json

Filter by Category

bash
uv run .claude/skills/development/skill-eval/scripts/eval-skills.py --category development

Summary Only

bash
uv run .claude/skills/development/skill-eval/scripts/eval-skills.py --summary-only

Best Practices

  • Run after creating new skills with /skill-creator to validate structure
  • Use --format json in CI pipelines for machine-readable output
  • Address critical issues first (missing frontmatter, invalid YAML)
  • Use --severity warning to focus on actionable improvements
  • Run --category filters for focused audits of specific skill areas

Error Handling

Exit Code Meaning
0 All skills pass (no critical issues)
1 Critical failures found
2 Script error (missing directory, invalid arguments)
Common Issue Cause Fix
frontmatter_missing Skill uses legacy heading format Add --- delimited YAML frontmatter
yaml_invalid Syntax error in frontmatter Fix YAML syntax (check colons, indentation)
related_skill_unresolved Referenced skill name doesn't exist Correct the name or remove the reference
section_missing Missing required H2 section Add the section heading and content

Metrics & Success Criteria

Metric Target
Skills passing all critical checks 100%
Skills with complete v2 sections >80%
Resolved cross-references 100%
No TODO/FIXME in skills 100%

Version History

  • 1.0.0 (2026-01-29): Initial release with 18 checks, human + JSON output, category/skill filtering

Recommended Agent Skills

Expand your agent's capabilities with these related and highly-rated skills.

majiayu000/claude-skill-registry

agent-ops-spec

Manage specification documents in .agent/specs/. Use when user provides requirements, acceptance criteria, or feature descriptions that need to be tracked and validated against implementation.

163 31
Explore
majiayu000/claude-skill-registry

agent-ops-state

Maintain .agent state files. Use at session start, after meaningful steps, and before concluding: read/update constitution/memory/focus/issues/baseline consistently.

163 31
Explore
majiayu000/claude-skill-registry

agent-ops-spec

Manage specification documents in .agent/specs/. Use when user provides requirements, acceptance criteria, or feature descriptions that need to be tracked and validated against implementation.

163 31
Explore
majiayu000/claude-skill-registry

agent-ops-testing

Test strategy, execution, and coverage analysis. Use when designing tests, running test suites, or analyzing test results beyond baseline checks.

163 31
Explore
majiayu000/claude-skill-registry

agent-ops-testing

Test strategy, execution, and coverage analysis. Use when designing tests, running test suites, or analyzing test results beyond baseline checks.

163 31
Explore
majiayu000/claude-skill-registry

agent-ops-state

Maintain .agent state files. Use at session start, after meaningful steps, and before concluding: read/update constitution/memory/focus/issues/baseline consistently.

163 31
Explore

Didn't find tool you were looking for?

Be as detailed as possible for better results