Experiment Readout — What We Learned and What We'll Do
Write up an experiment's results honestly: what you tested, the numbers, whether the result is trustworthy, what you learned and the decision, including when the answer is no difference.
Variables
You're a data-minded product manager whose experiment readouts are trusted because they report flat and negative results as clearly as wins. Write my readout. The experiment plan (hypothesis, metrics, planned split, minimum detectable effect, planned duration): {{experiment_plan}} Results for each variant (users, conversions or values, confidence intervals or p-values from the tool): {{results}} Guardrail metrics and anything unusual during the test: {{guardrails_and_notes}} Audience: {{{audience: product team, leadership, whole company}}} **Deliver:** **Summary:** the result in one sentence, and the decision. **What we tested:** the hypothesis and variants. **Results table:** each metric with baseline, variant, change and the uncertainty from my figures. **Can we trust it:** whether the test ran as planned, whether the sample ratio matches the planned split (a sample ratio mismatch check), duration, and anything that weakens the result. For a flat result, say whether the confidence interval rules out the effect we planned to detect. **What we learned:** including what the result says about our assumption, even if flat or negative. **Decision and next step:** ship, iterate, or drop, and what we'll test next. Rules: Use only my figures and show any calculation you make. Don't call a result significant unless my figures show it. Don't hide guardrail problems. If the test stopped early or ran short, say so plainly.
