Sample Size, Minimum Detectable Effect, and Power › Worked Example: Clinical Trial Design
Michael BrenndoerferJanuary 7, 20265 min read
Machine Learning from ScratchAll chapters Part 2 of 9Hypothesis Testing
Sample Size, Minimum Detectable Effect, and Power › Worked Example: Clinical Trial DesignOne section · Sections in this chapter7 / 12 ↓
Sections in this chapter12
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
Let's work through a complete power analysis for a realistic scenario.
Scenario: A pharmaceutical company is designing a clinical trial to test whether a new blood pressure medication reduces systolic blood pressure more than the current standard treatment. Based on pilot data:
- Current treatment reduces systolic BP by 10 mmHg on average
- Standard deviation of BP reduction is approximately 15 mmHg
- The company wants to detect a 5 mmHg additional reduction (clinically meaningful)
- They want 90% power to ensure reliable results
- Using standard α = 0.05 (two-tailed)
In[13]:
Codeimport numpy as np
from scipy import stats
# Study parameters
current_treatment_effect = 10 # mmHg reduction
new_treatment_effect = 15 # Expected mmHg reduction
effect_size = new_treatment_effect - current_treatment_effect # 5 mmHg
sigma = 15 # Standard deviation
alpha = 0.05
target_power = 0.90
print("=== Clinical Trial Power Analysis ===\n")
print("Parameters:")
print(f" Expected effect: {effect_size} mmHg improvement over standard")
print(f" Standard deviation: {sigma} mmHg")
print(f" Significance level: α = {alpha}")
print(f" Target power: {target_power * 100}%")
# Calculate Cohen's d
d = effect_size / sigma
print(f"\nStandardized effect size (Cohen's d): {d:.3f}")
# Sample size calculation
z_alpha = stats.norm.ppf(1 - alpha / 2)
z_beta = stats.norm.ppf(target_power)
n_per_group = 2 * ((z_alpha + z_beta) / d) ** 2
n_per_group = int(np.ceil(n_per_group))
print("\n=== Required Sample Size ===")
print(f" z_α/2 = {z_alpha:.4f}")
print(f" z_(1-β) = {z_beta:.4f}")
print(f" n per group = 2 × (({z_alpha:.3f} + {z_beta:.3f}) / {d:.3f})²")
print(f" n per group = {n_per_group}")
print(f" Total participants = {2 * n_per_group}")Out[13]:
Console=== Clinical Trial Power Analysis === Parameters: Expected effect: 5 mmHg improvement over standard Standard deviation: 15 mmHg Significance level: α = 0.05 Target power: 90.0% Standardized effect size (Cohen's d): 0.333 === Required Sample Size === z_α/2 = 1.9600 z_(1-β) = 1.2816 n per group = 2 × ((1.960 + 1.282) / 0.333)² n per group = 190 Total participants = 380
In[14]:
Code# Sensitivity analysis: What if our assumptions are wrong?
print("\n=== Sensitivity Analysis ===\n")
print("What if the true effect or variability differs from our assumptions?\n")
# Vary effect size
print("1. Effect Size Sensitivity (σ = 15 mmHg, power = 90%):")
print(f" {'Effect (mmHg)':<15} {'Cohen d':<10} {'n per group':<15}")
print(" " + "-" * 40)
for effect in [3, 4, 5, 6, 7]:
d = effect / sigma
n = 2 * ((z_alpha + z_beta) / d) ** 2
print(f" {effect:<15} {d:<10.3f} {int(np.ceil(n)):<15}")
# Vary standard deviation
print("\n2. Variance Sensitivity (effect = 5 mmHg, power = 90%):")
print(f" {'SD (mmHg)':<15} {'Cohen d':<10} {'n per group':<15}")
print(" " + "-" * 40)
for sd in [10, 12, 15, 18, 20]:
d = effect_size / sd
n = 2 * ((z_alpha + z_beta) / d) ** 2
print(f" {sd:<15} {d:<10.3f} {int(np.ceil(n)):<15}")
# Vary power
print("\n3. Power Sensitivity (effect = 5 mmHg, σ = 15 mmHg):")
print(f" {'Power':<15} {'n per group':<15} {'Total N':<15}")
print(" " + "-" * 45)
for power in [0.70, 0.80, 0.85, 0.90, 0.95]:
z_b = stats.norm.ppf(power)
n = 2 * ((z_alpha + z_b) / d) ** 2
n = int(np.ceil(n))
print(f" {power * 100:.0f}%{'':<10} {n:<15} {2 * n:<15}")Out[14]:
Console=== Sensitivity Analysis === What if the true effect or variability differs from our assumptions? 1. Effect Size Sensitivity (σ = 15 mmHg, power = 90%): Effect (mmHg) Cohen d n per group ---------------------------------------- 3 0.200 526 4 0.267 296 5 0.333 190 6 0.400 132 7 0.467 97 2. Variance Sensitivity (effect = 5 mmHg, power = 90%): SD (mmHg) Cohen d n per group ---------------------------------------- 10 0.500 85 12 0.417 122 15 0.333 190 18 0.278 273 20 0.250 337 3. Power Sensitivity (effect = 5 mmHg, σ = 15 mmHg): Power n per group Total N --------------------------------------------- 70% 198 396 80% 252 504 85% 288 576 90% 337 674 95% 416 832
Out[15]:
Visualization
Decision Summary
In[16]:
Codeprint("=== Clinical Trial Design Decision ===\n")
n_required = 190 # From our calculation
dropout_rate = 0.15 # Typical for clinical trials
n_enrolled = int(np.ceil(n_required / (1 - dropout_rate)))
print("Recommended Design:")
print(f" • Enroll {n_enrolled} patients per group ({2 * n_enrolled} total)")
print(f" (Accounts for {dropout_rate * 100:.0f}% expected dropout)")
print(f" • Expected completers: {n_required} per group")
print(" • Power: 90% to detect 5 mmHg difference")
print(" • MDE at enrolled sample: ~4.3 mmHg")
print()
print("Study Characteristics:")
print(" • Can detect effects as small as Cohen's d = 0.29")
print(" • 10% risk of missing a true 5 mmHg effect (Type II)")
print(" • 5% risk of false positive (Type I)")
print()
print("Practical Considerations:")
print(" • Budget for ~450 total enrolled patients")
print(" • Allow for longer recruitment if dropout > expected")
print(" • Consider adaptive design if early results warrant")Out[16]:
Console=== Clinical Trial Design Decision ===
Recommended Design:
• Enroll 224 patients per group (448 total)
(Accounts for 15% expected dropout)
• Expected completers: 190 per group
• Power: 90% to detect 5 mmHg difference
• MDE at enrolled sample: ~4.3 mmHg
Study Characteristics:
• Can detect effects as small as Cohen's d = 0.29
• 10% risk of missing a true 5 mmHg effect (Type II)
• 5% risk of false positive (Type I)
Practical Considerations:
• Budget for ~450 total enrolled patients
• Allow for longer recruitment if dropout > expected
• Consider adaptive design if early results warrant