Michael Brenndoerfer | Data & AI, Private Equity, Technology

Michael Brenndoerfer

Sample Size, Minimum Detectable Effect, and Power › Visualizing Power Analysis

Michael BrenndoerferJanuary 7, 20264 min read

Machine Learning from ScratchAll chapters

Part 2 of 9Hypothesis Testing

Chapter 19Sample Size, Minimum Detectable Effect, and Power › Visualizing Power Analysis

Reading section 6 of 12 · Visualizing Power Analysis · 4 minSection 6 of 12 · 4 min

Section 6 of 12 · 4 min read

Sample Size, Minimum Detectable Effect, and Power › Visualizing Power AnalysisOne section · Sections in this chapter6 / 12 ↓

Sections in this chapter12

Reading tools Definitions & inline links

Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.

Article links

Make inline references clickable

Understanding how the five quantities interact is easier with visualization.

Out[10]:

Visualization

Heatmap showing required sample size for different effect sizes and power levels. — Required sample size as a function of effect size and power. Smaller effect sizes and higher power requirements demand dramatically larger samples. The white contour lines show specific sample size thresholds. Note how requirements escalate rapidly when trying to detect small effects (d < 0.3) with high power.

Out[11]:

Visualization

Power versus effect size for 25 observations per group, with 80 percent power, MDE, and conventional effect-size reference lines. — Power curve for n = 25 per group (MDE = 0.79).

Power versus effect size for 50 observations per group, with 80 percent power, MDE, and conventional effect-size reference lines. — Power curve for n = 50 per group (MDE = 0.56).

Out[12]:

Visualization

Power versus effect size for 100 observations per group, with 80 percent power, MDE, and conventional effect-size reference lines. — Power curve for n = 100 per group (MDE = 0.40).

Power versus effect size for 200 observations per group, with 80 percent power, MDE, and conventional effect-size reference lines. — Power curve for n = 200 per group (MDE = 0.28).