ReddRadar
Agent skill
measure-experiment-results
Documents the results of a completed experiment or A/B test with statistical analysis, learnings, and recommendations. Use after experiments conclude to communicate findings, inform decisions, and build organizational knowledge.
Install this agent skill to your Project
npx add-skill https://github.com/product-on-purpose/pm-skills/tree/main/skills/measure-experiment-results
Metadata
Additional technical details for this skill
- author
- product-on-purpose
- category
- reflection
- frameworks
-
[ "triple-diamond", "lean-startup", "design-thinking" ]
SKILL.md
Experiment Results
An experiment results document captures what happened when you tested a hypothesis, including statistical outcomes, segment analysis, learnings, and clear recommendations. Good results documentation turns individual experiments into organizational knowledge that improves future decision-making.
When to Use
- After an A/B test or experiment reaches statistical significance
- When an experiment is ended early (for any reason)
- To communicate findings to stakeholders who weren't involved
- During decision-making about whether to ship, iterate, or kill a feature
- To build a repository of learnings that inform future experiments
Instructions
When asked to document experiment results, follow these steps:
-
Summarize the Experiment Provide context: what was tested, when it ran, how much traffic it received. Link to the original experiment design document if one exists.
-
Restate the Hypothesis Remind readers what you believed would happen and why. This frames the results interpretation.
-
Present Primary Results Show the primary metric outcome clearly: what were the values for control and treatment? Include statistical significance (p-value), confidence intervals, and sample sizes. Be honest about whether results are conclusive.
-
Analyze Secondary Metrics Present guardrail metrics that ensure you didn't cause unintended harm. Note any secondary metrics that moved unexpectedly—both positive and negative.
-
Segment the Data Look for differential effects across user segments (platform, tenure, plan type, etc.). Sometimes overall results mask important segment-level insights.
-
Extract Learnings What did you learn beyond the numbers? Include surprising findings, questions raised, and implications for the product hypothesis. Negative results are valuable learnings.
-
Make a Recommendation Be clear: should we ship, iterate, or kill? Support the recommendation with the evidence. If the decision is nuanced, explain the trade-offs.
-
Define Next Steps Specify what happens now—engineering work to ship, follow-up experiments, metrics to continue monitoring, or documentation to update.
Output Format
Use the template in references/TEMPLATE.md to structure the output.
Quality Checklist
Before finalizing, verify:
- Statistical methods and significance are clearly stated
- Confidence intervals are included (not just p-values)
- Segment analysis checked for differential effects
- Secondary/guardrail metrics are reported
- Learnings go beyond just the numbers
- Recommendation is clear and actionable
- Negative or inconclusive results are reported honestly
Examples
See references/EXAMPLE.md for a completed example.
Recommended Agent Skills
Expand your agent's capabilities with these related and highly-rated skills.
product-on-purpose/pm-skills
init-project
Initialize projects with agentic coding structure. Use when setting up a new project, adding AI agent support to existing project, or when user says "init", "initialize", "setup project", or "scaffold". Creates AGENTS folder, documentation templates, and _NOTES scratch space.
product-on-purpose/pm-skills
init-project-jpkb
Initialize new JPKB projects with standardized documentation and folder structure. JPKB-specific version with category folders and fixed base path. Use when creating a new project in the jpkb repository, when the user says "init project", "new project", or when the target is the JPKB projects folder.
product-on-purpose/pm-skills
wrap-session
End-of-session documentation workflow that updates README, CHANGELOG, agent context files, and creates session logs. Use when wrapping up a working session, when asked to document session progress, when preparing handoff documentation, or when the user says "wrap up", "end session", "document progress", or "save session".
product-on-purpose/pm-skills
skill-template
product-on-purpose/pm-skills
utility-update-pm-skills
Checks for newer pm-skills releases, compares local vs. latest version, previews what would change, and updates local files after user confirmation. Generates a structured update report documenting changed files, new capabilities, and the value delta between versions. Use when you want to bring a local pm-skills installation up to date.
product-on-purpose/pm-skills
utility-update-pm-skills
Validates internet access, compares the locally installed pm-skills version against the latest public release, and updates local files with conflict-aware overwrite-or-skip options. Produces an update report listing changed files, skipped files, and new capabilities. Use when you want to bring a local pm-skills installation up to date.
Didn't find tool you were looking for?