Effect Sizes and Statistical Significance › Best Practices for Reporting Effect Sizes
Michael BrenndoerferJanuary 8, 20262 min read
Machine Learning from ScratchAll chapters Part 2 of 9Hypothesis Testing
Effect Sizes and Statistical Significance › Best Practices for Reporting Effect SizesOne section · Sections in this chapter8 / 11 ↓
Sections in this chapter11
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
What to Report
- Always report effect sizes alongside p-values
- Include confidence intervals when possible
- Use the appropriate effect size for your design
- Interpret in context
APA Style Reporting
In[15]:
Codedef apa_t_test_report(group1, group2, alpha=0.05):
"""Generate APA-style report for independent samples t-test."""
n1, n2 = len(group1), len(group2)
t_stat, p_val = stats.ttest_ind(group2, group1)
d = cohens_d(group2, group1)
d, ci_low, ci_high = cohens_d_ci(group2, group1)
# Two-tailed
sig = "p < .001" if p_val < 0.001 else f"p = {p_val:.3f}"
report = f"""
APA-Style Report:
================
The treatment group (M = {np.mean(group2):.2f}, SD = {np.std(group2, ddof=1):.2f})
showed {"significantly" if p_val < alpha else "no significant"} different scores than
the control group (M = {np.mean(group1):.2f}, SD = {np.std(group1, ddof=1):.2f}),
t({n1 + n2 - 2}) = {abs(t_stat):.2f}, {sig}.
The effect size was {"large" if abs(d) >= 0.8 else "medium" if abs(d) >= 0.5 else "small"},
d = {d:.2f}, 95% CI [{ci_low:.2f}, {ci_high:.2f}].
"""
return report
# Example
np.random.seed(42)
control = np.random.normal(50, 10, 40)
treatment = np.random.normal(58, 10, 40)
print(apa_t_test_report(control, treatment))Out[15]:
ConsoleAPA-Style Report: ================ The treatment group (M = 57.71, SD = 9.65) showed significantly different scores than the control group (M = 47.81, SD = 9.53), t(78) = 4.62, p < .001. The effect size was large, d = 1.03, 95% CI [0.56, 1.51].
Summary Table Format
| Measure | Control | Treatment | Effect Size | 95% CI | Interpretation |
|---|---|---|---|---|---|
| Test Score | 50.2 ± 10.1 | 58.3 ± 9.8 | d = 0.81 | [0.35, 1.27] | Large effect |