Michael Brenndoerfer | Data & AI, Private Equity, Technology

Michael Brenndoerfer

Effect Sizes and Statistical Significance › Variants of Cohen's d

Michael BrenndoerferJanuary 8, 20262 min read

Machine Learning from ScratchAll chapters

Part 2 of 9Hypothesis Testing

Chapter 20Effect Sizes and Statistical Significance › Variants of Cohen's d

Reading section 4 of 11 · Variants of Cohen's d · 2 minSection 4 of 11 · 2 min

Section 4 of 11 · 2 min read

Effect Sizes and Statistical Significance › Variants of Cohen's dOne section · Sections in this chapter4 / 11 ↓

Sections in this chapter11

Reading tools Definitions & inline links

Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.

Article links

Make inline references clickable

Several variations of Cohen's d exist for different situations:

Hedges' g: Correcting for Small Sample BiasLink Copied

Cohen's d has a slight upward bias in small samples. Hedges' g applies a correction:

g = d \times \left(1 - \frac{3}{4(n_1 + n_2) - 9}\right)

In[5]:

Code

def hedges_g(group1, group2):
    """
    Calculate Hedges' g (bias-corrected Cohen's d).
    """
    n1, n2 = len(group1), len(group2)
    d = cohens_d(group1, group2)

    # Correction factor
    correction = 1 - (3 / (4 * (n1 + n2) - 9))

    return d * correction


# Compare d and g for different sample sizes
print("Comparison of Cohen's d and Hedges' g:\n")
print(
    f"{'n per group':<15} {'Cohen d':<15} {'Hedges g':<15} {'Difference':<15}"
)
print("-" * 60)

np.random.seed(42)
for n in [10, 20, 50, 100]:
    g1 = np.random.normal(0, 1, n)
    g2 = np.random.normal(0.5, 1, n)

    d = cohens_d(g2, g1)
    g = hedges_g(g2, g1)

    print(f"{n:<15} {d:<15.3f} {g:<15.3f} {(d - g) * 100:.1f}%")

print("\nNote: The correction becomes negligible for n > 50")

Out[5]:

Console

Comparison of Cohen's d and Hedges' g:

n per group     Cohen d         Hedges g        Difference     
------------------------------------------------------------
10              -0.999          -0.957          -4.2%
20              0.824           0.807           1.6%
50              0.556           0.552           0.4%
100             0.374           0.372           0.1%

Note: The correction becomes negligible for n > 50

Glass's Delta: Unequal VariancesLink Copied

When the treatment might change variability, use the control group's standard deviation as the denominator:

\Delta = \frac{\bar{x}_1 - \bar{x}_2}{s_{\text{control}}}

In[6]:

Code

def glass_delta(treatment, control):
    """
    Calculate Glass's delta using only control group SD.
    Useful when treatment might change variability.
    """
    return (np.mean(treatment) - np.mean(control)) / np.std(control, ddof=1)


# Example where treatment increases variability
np.random.seed(42)
control = np.random.normal(50, 10, 40)
treatment = np.random.normal(55, 15, 40)  # Same mean diff, but more variable

d = cohens_d(treatment, control)
delta = glass_delta(treatment, control)

print("Treatment that increases variability:")
print(
    f"  Control: mean = {np.mean(control):.1f}, SD = {np.std(control, ddof=1):.1f}"
)
print(
    f"  Treatment: mean = {np.mean(treatment):.1f}, SD = {np.std(treatment, ddof=1):.1f}"
)
print(f"\n  Cohen's d: {d:.3f}")
print(f"  Glass's Δ: {delta:.3f}")
print("\nGlass's Δ uses only control SD, giving a cleaner baseline comparison")

Out[6]:

Console

Treatment that increases variability:
  Control: mean = 47.8, SD = 9.5
  Treatment: mean = 54.6, SD = 14.5

  Cohen's d: 0.551
  Glass's Δ: 0.709

Glass's Δ uses only control SD, giving a cleaner baseline comparison