Sample Size, Minimum Detectable Effect, and Power › Summary
Machine Learning from ScratchAll chapters Sample Size, Minimum Detectable Effect, and Power › SummaryOne section · Sections in this chapter10 / 12 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
Power analysis is essential for designing studies that can answer research questions. The key points are:
The five quantities (sample size, effect size, variance, α, power) are mathematically connected. Given any four, you can solve for the fifth.
Sample size formulas vary by test type but share a common structure. The key insight is that requirements scale with the square of the ratio (z_α + z_β)/d, so small changes in effect size dramatically affect sample needs.
Minimum detectable effect (MDE) tells you the smallest effect your study can reliably detect. Use it to assess whether a study is worth conducting.
Underpowered studies have serious consequences: high false negative rates, inflated effect estimates (winner's curse), and poor replicability.
Power analysis should be done before data collection, not after. Post-hoc power analysis is meaningless because it's just a transformation of the p-value.
Sensitivity analysis is important because effect size and variance assumptions are often uncertain. Understand how your conclusions change if assumptions are wrong.