Audit Scoring

Purpose

This directory provides the quantitative scoring framework for measuring audit quality. Use these metrics to objectively evaluate performance, track improvement over time, and identify areas needing attention.

Available Files

Core Metrics at a Glance

Composite Score = (0.35 × Detection) + (0.25 × Precision) + (0.15 × Severity) + (0.15 × Coverage) + (0.10 × Efficiency)

Severity Weights for Efficiency Scoring

These weights connect the scoring system to the severity classification:

How to Use

1. After an audit → Fill out the Score Card Template in AUDIT_SCORING.md
2. Classify findings → Use severity/ files + severity-scoring decision tree
3. Track monthly → Use the Monthly Score Tracking template
4. Identify gaps → Category-specific scores highlight weak areas
5. Improve → Low category scores → update checklists and patterns

Related Skills

• Severity Classification — HIGH / MEDIUM / LOW / GAS finding databases
• Severity Scoring Decision Tree — How to assign severity levels
• Feedback Loop — Scores feed back into skill improvement
• Audit Report Templates — Report structure with severity sections
• Prompt Evolution — Higher-scoring prompts get promoted

Prerequisites

Scoring requires completed audit findings with severity classifications. The Severity Classification skill MUST be applied before scoring.

Validation

To verify scoring accuracy, compare computed composite scores against known benchmarks:

# Example composite score calculation
detection = 0.85   # 85% of real vulns found
precision = 0.80   # 80% valid findings
severity_acc = 0.90 # 90% correct severity
coverage = 0.75    # 75% codebase covered
efficiency = 0.70  # Weighted findings per hour

composite = (0.35 * detection + 0.25 * precision + 0.15 * severity_acc + 0.15 * coverage + 0.10 * efficiency)
print(f"Composite Score: {composite:.2f}")  # Expected: 0.81

# Score thresholds for audit quality tiers
tiers:
  elite: 0.90+       # Top-tier competitive auditor
  proficient: 0.75+  # Solid professional auditor
  developing: 0.60+  # Learning auditor
  needs_work: <0.60  # Consider additional training

# Validate scoring data integrity
python scripts/quality-check.py skills/scoring/SKILL.md

Behavior Guidelines

• Detection and Precision scores are required for every engagement
• Coverage tracking is optional for quick scans but MUST be included in full audits
• Efficiency scoring should be used for self-improvement, never to rush audits

References

• Scoring References - Industry benchmarks and calibration data