Podcast Show Notes & Episode Outlines — AI-Written, Publish-Ready

# Episode 47: "Why Most A/B Tests Fail"
**Guest:** Dr. Sarah Chen — Head of Experimentation, GrowthLab
**Duration:** 42 min | **Season:** 3, Episode 12
---
## Episode Summary
Sarah breaks down the three statistical mistakes that invalidate 60% of A/B tests — and shares the framework her team uses to run reliable experiments.
## Timestamps
- 00:00 — Intro & why this topic matters
- 02:15 — The "peeking problem" — why checking early ruins your test
- 08:40 — Sample size calculators: which ones work, which lie
- 14:30 — Multi-variant testing pitfalls
- 22:10 — Sarah's 5-step experiment framework
- 31:45 — Rapid fire: tools, books, and hot takes
## Key Takeaways
- **Stop peeking:** Checking results before statistical significance inflates false positives to 30%+
- **Pre-register your hypothesis:** Define success metrics before launching
- **Run power analysis first:** Most tests are underpowered — calculate minimum sample size upfront
## Resources Mentioned
- Book: *Trustworthy Online Controlled Experiments* — Kohavi, Tang & Xu
- Tool: Evan Miller's sample size calculator (evanmiller.org)
## SEO Description
Dr. Sarah Chen explains the three statistical mistakes that invalidate most A/B tests — and shares a 5-step framework for experiments you can trust.