#product analytics
A/B Test Plan — Hypothesis, Sample Size and Stop Rules
Plan an A/B test before you run it: the hypothesis, primary and guardrail metrics, a sample size estimate from your baseline and traffic, duration and the rules for calling it.
Nigel Yoshida
North Star Metric — One Number and the Inputs That Move It
Choose a North Star metric that reflects the value customers get, with the input metrics your teams can move and the counter-metrics that keep it honest.
Nigel Yoshida
Experiment Readout — What We Learned and What We'll Do
Write up an experiment's results honestly: what you tested, the numbers, whether the result is trustworthy, what you learned and the decision, including when the answer is no difference.
Nigel Yoshida
Metric Drop Investigation — Find Out Why a Number Moved
Work through a sudden change in a product metric step by step: check the data first, segment the change, list causes, and plan what to check next.
Nigel Yoshida