PR Distill Skill

<ROLE>PR Review Analyst. Your reputation depends on accurately identifying which changes need human review and which are safe to skip.</ROLE>

Analyzes pull requests to categorize changes by review necessity, reducing cognitive load on large PRs.

Invariant Principles

1. Heuristics First, AI Second: Always run heuristic pattern matching before invoking AI analysis. Heuristics are fast and deterministic.
2. Confidence Requires Evidence: Never mark a change as "safe to skip" without a pattern match or AI explanation justifying the confidence level.
3. Surface Uncertainty: When confidence is low, categorize as "uncertain" rather than guessing. Humans should decide ambiguous cases.
4. Preserve Context: The report must include enough diff context for reviewers to understand changes without switching to the PR itself.

MCP Tools

Execution Flow

This skill uses a two-phase execution model where the agent orchestrates MCP tool calls:

Phase 1: Fetch, Parse, Match

# Step 1: Fetch PR data
pr_data = pr_fetch("<pr-identifier>")

# Step 2: Parse the diff
diff_result = pr_diff(pr_data["diff"])

# Step 3: Match patterns against files
match_result = pr_match_patterns(
    files=diff_result["files"],
    project_root="/path/to/project"
)

This produces:

• match_result["matched"]: Files with pattern matches (categorized)
• match_result["unmatched"]: Files requiring AI analysis

Phase 2: AI Analysis (if needed)

For unmatched files, analyze each to determine:

• review_required: Significant logic, API, or behavior changes
• safe_to_skip: Formatting, comments, trivial refactors
• uncertain: When confidence is low, surface for human decision

Phase 3: Generate Report

Produce a markdown report with:

1. Summary of changes by category
2. Full diffs for review_required items
3. Pattern matches with confidence levels
4. Discovered patterns with bless commands

Usage

When invoked via /distilling-prs <pr>:

1. Call pr_fetch("<pr>") to get PR data
2. Call pr_diff(pr_data["diff"]) to parse the diff
3. Call pr_match_patterns(files, project_root) to run heuristics
4. For unmatched files, analyze with AI to categorize
5. Generate and present the report

Examples

# Analyze PR by number (uses current repo context)
pr_data = pr_fetch("123")

# Analyze PR by URL
pr_data = pr_fetch("https://github.com/owner/repo/pull/123")

# Parse the diff
diff_result = pr_diff(pr_data["diff"])

# Run pattern matching
match_result = pr_match_patterns(
    files=diff_result["files"],
    project_root="/Users/alice/project"
)

# Bless a discovered pattern
pr_bless_pattern("/Users/alice/project", "query-count-json")

# List all patterns
patterns = pr_list_patterns("/Users/alice/project")

Configuration

Config file: ~/.local/spellbook/docs/<project-encoded>/distilling-prs-config.json

{
  "blessed_patterns": ["query-count-json", "import-cleanup"],
  "always_review_paths": ["**/migrations/**", "**/permissions.py"],
  "query_count_thresholds": {
    "relative_percent": 20,
    "absolute_delta": 10
  }
}

Builtin Patterns

The skill includes 15 builtin patterns across confidence levels:

Always Review (5): migration files, permission changes, model changes, signal handlers, endpoint changes

High Confidence (5): settings changes, query count JSON, debug print statements, import cleanup, gitignore updates

Medium Confidence (5): backfill commands, decorator removals, factory setup, test renames, test assertion updates

Use pr_list_patterns() to see all patterns with their IDs and descriptions.