Definition
Significance quantifies the probability that observed differences between variants are real. 95% confidence still means one in twenty 'winners' is noise - and stopping tests at first significance inflates false positives further.
Why it matters
Acting on noise feels like velocity and performs like gambling; significance discipline separates experimentation programs from superstition.
Benchmarks
Pre-commit to 95% confidence and minimum sample before launch
All glossary terms