Agent skill
experiment-workflow
How to create, maintain and extend Python experiments. Experiment lifecycle workflows and best practices.
Install this agent skill to your Project
npx add-skill https://github.com/majiayu000/claude-skill-registry/tree/main/skills/other/experiment-workflow-joernstoehler-msc-viterbo-2
SKILL.md
Experiment Workflow
This skill covers the complete lifecycle of Python experiments in experiments/. Experiments are the primary way this thesis generates empirical evidence.
Source of truth: filesystem → experiments/CLAUDE.md → this skill
Project Context
File Layout
| Artifact | Location |
|---|---|
| Tracking | Task file in tasks/ |
| Code | experiments/<label>/ |
| SPEC.md | experiments/<label>/SPEC.md |
| Tests | experiments/<label>/test_*.py (colocated) |
| Data | data/<label>/ |
| Thesis assets | thesis/assets/<label>/ |
Standard Code Structure
experiments/<label>/
├── SPEC.md # Research question, method, success criteria
├── stage_build.py # Generate data
├── test_stage_build.py # Colocated test
├── stage_analyze.py # Compute results
└── stage_plot.py # Create figures
Why stages? Separate expensive computation (build) from cheap iteration (analyze/plot). Re-run analysis without regenerating data.
Teaching Example
experiments/_example/ demonstrates all conventions. Study it first when creating a new experiment.
Experiment Lifecycle
Initial Development
| Phase | Owner | Output |
|---|---|---|
| Ideation | Jörn + PM agent | Task file in tasks/inbox/ |
| Planning | Task agent (plan mode) | SPEC.md with method and success criteria |
| Development | Task agent | stage_* code, FINDINGS.md, artifacts, PR |
| Code review | Review agent | Approved PR (or escalate back) |
| Merge | PM agent | Merged PR, task moved to done/ |
Ongoing Maintenance
After initial development, experiments may be rerun when the codebase changes.
| Phase | Owner | Output |
|---|---|---|
| Rerun | Any agent | Updated data and asset artifacts |
| Maintenance | Any agent | Updated FINDINGS.md; escalate if needed |
Invariants
SPEC.md Template
# <label> Experiment
**Status:** Ideation / Specified / In Progress / Complete
## Research Question
What are we trying to learn?
## Method
How will we answer the question?
## Success Criteria
What outcome means "we are satisfied"?
## Expected Outputs
- data/<label>/...
- thesis/assets/<label>/...
FINDINGS.md Template
# <label> Findings
## Results Summary
Key findings from the data.
## Interpretation
What the results mean for the research question.
## Escalation Procedures
When and how to flag issues.
## Changelog
- YYYY-MM-DD: Initial findings
- YYYY-MM-DD: Updated after <change>
FINDINGS.md Content Guidelines
FINDINGS.md is a living document updated on each rerun:
- Results summary: Key findings from the data
- Interpretation: What the results mean for the research question
- Escalation procedures: When and how to flag issues (e.g., "if validation fails, open issue before updating thesis")
- Changelog: Dated entries when findings change significantly
Stage Patterns
stage_build.py
"""Generate data for <label> experiment."""
from pathlib import Path
# All config hardcoded for reproducibility
CONFIG = {
"experiment_name": "<label>",
"variant": "full", # Change to "smoke" for quick runs
}
DATA_DIR = Path(__file__).parent.parent.parent / "data"
OUTPUT_DIR = DATA_DIR / CONFIG["experiment_name"]
def main():
OUTPUT_DIR.mkdir(parents=True, exist_ok=True)
# Generate data
# Save to OUTPUT_DIR / "results.json" or similar
if __name__ == "__main__":
main()
stage_plot.py
"""Generate plots for <label> experiment."""
from pathlib import Path
import matplotlib.pyplot as plt
CONFIG = {"experiment_name": "<label>"}
DATA_DIR = Path(__file__).parent.parent.parent / "data"
INPUT_DIR = DATA_DIR / CONFIG["experiment_name"]
ASSETS_DIR = Path(__file__).parent.parent.parent / "thesis" / "assets" / CONFIG["experiment_name"]
def main():
ASSETS_DIR.mkdir(parents=True, exist_ok=True)
# Load data
# Create figures
plt.savefig(ASSETS_DIR / "figure.pdf")
if __name__ == "__main__":
main()
Escalation Triggers
Escalate (update task file and notify Jörn) when:
- Validation tests that previously passed now fail
- Data artifacts contradict FINDINGS.md conclusions
- Upstream code changes invalidate assumptions in SPEC.md
Philosophy
- Task files track progress - coordination, blockers, status updates
- Files contain research content - SPEC.md, FINDINGS.md, stages, data
- Don't compress ideas - preserve intellectual labor in SPEC/FINDINGS
- Reproduction must be obvious - pattern self-evident from repo structure
- Escalate don't hide - if results change qualitatively, flag it rather than silently updating
Code Style
- KISS (Keep It Simple)
- Pure functions (closeness to math)
- Fast dev cycle (no external users)
Recommended Agent Skills
Expand your agent's capabilities with these related and highly-rated skills.
agent-ops-spec
Manage specification documents in .agent/specs/. Use when user provides requirements, acceptance criteria, or feature descriptions that need to be tracked and validated against implementation.
agent-ops-state
Maintain .agent state files. Use at session start, after meaningful steps, and before concluding: read/update constitution/memory/focus/issues/baseline consistently.
agent-ops-spec
Manage specification documents in .agent/specs/. Use when user provides requirements, acceptance criteria, or feature descriptions that need to be tracked and validated against implementation.
agent-ops-testing
Test strategy, execution, and coverage analysis. Use when designing tests, running test suites, or analyzing test results beyond baseline checks.
agent-ops-testing
Test strategy, execution, and coverage analysis. Use when designing tests, running test suites, or analyzing test results beyond baseline checks.
agent-ops-state
Maintain .agent state files. Use at session start, after meaningful steps, and before concluding: read/update constitution/memory/focus/issues/baseline consistently.
Didn't find tool you were looking for?