Detecting AI-Generated Code

Overview

Systematic framework for detecting AI-generated and AI-assisted code. Uses tiered signal detection with confidence scoring.

Core principle: Check metadata FIRST (highest confidence), then comments, then code patterns. Multiple signals compound confidence.

Detection Framework (Check in Order)

Tier 1: Metadata Signals (Highest Confidence)

ALWAYS CHECK THESE FIRST - they're definitive.

AI README Template Pattern:

# Project Name
## Overview
[Formal description]
## Features
- Feature 1
- Feature 2
## Installation
[Perfect instructions]
## API Endpoints (table)
## Environment Variables (table)
## Contributing
## License

If README follows this EXACT structure with perfect tables and no personality → High AI probability.

AI Commit Message Patterns:

Add user authentication with JWT tokens
Add password recovery functionality
Add comprehensive documentation      ← "comprehensive" is strong AI tell
Implement feature X
Update component Y
Fix bug in Z

Mechanical uniformity + "Add/Implement/Update/Fix" pattern → AI signal.

Human commit patterns (for contrast):

wip: payments
fix typo in readme
initial payment setup
refactor auth - cleanup
oops forgot the tests

Tier 2: Comment Patterns (High Confidence)

Verbatim AI Phrases:

• "This file contains..."
• "This function handles..."
• "Here we define..."
• "Now we check..."
• "Finally, we return..."

Tier 3: Code Structure (Medium Confidence)

Important: Clean code alone is NOT evidence of AI. Experienced developers also write clean code.

Tier 4: Anti-Signals (What Humans Do, AI Doesn't)

These DECREASE AI probability:

Confidence Scoring

Compound scoring: Multiple signals from different tiers = higher confidence than multiple signals from same tier.

AI-Assisted Detection (Partial AI Use)

Look for:

• Style jumps - Casual code with formal JSDoc blocks
• Git history jumps - "Add comprehensive documentation" as separate commit
• Quality inconsistency - Some files over-documented, others bare
• Misplaced docs - JSDoc on library calls, not wrapper functions
• Parameter mismatches - Docs describe different params than code has

This pattern: Human wrote code → Asked AI to add documentation.

Common Mistakes

Quick Reference Workflow

1. Check git log - Look for Co-Authored-By, uniform commit patterns
2. Check README - Template structure? Perfect tables? No personality?
3. Check file headers - Generated footers? "This file contains..." comments?
4. Check code comments - Narrating style? Over-documentation?
5. Check for anti-signals - TODOs? Shortcuts? Debug code?
6. Score confidence - Compound signals from multiple tiers
7. Report with evidence - List specific signals found

When NOT to Flag

• Clean, well-written code (without other signals)
• Standard patterns for solved problems (EventEmitter, validators)
• Good documentation (if style matches rest of codebase)
• Experienced developer output

Clean code without Tier-1 or Tier-2 signals = insufficient evidence.