Michael Brenndoerfer | Data & AI, Private Equity, Technology

Michael Brenndoerfer

The F-Test and F-Distribution › F-Test for Comparing Two Variances

Michael BrenndoerferJanuary 4, 20266 min read

Machine Learning from ScratchAll chapters

Part 2 of 9Hypothesis Testing

Chapter 16The F-Test and F-Distribution › F-Test for Comparing Two Variances

Reading section 4 of 10 · F-Test for Comparing Two Variances · 6 minSection 4 of 10 · 6 min

Section 4 of 10 · 6 min read

The F-Test and F-Distribution › F-Test for Comparing Two VariancesOne section · Sections in this chapter4 / 10 ↓

Sections in this chapter10

Reading tools Definitions & inline links

Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.

Article links

Make inline references clickable

The most direct application of the F-test compares variances between two independent populations.

Hypotheses and Test StatisticLink Copied

Null hypothesis: $H_0: \sigma_1^2 = \sigma_2^2$ (population variances are equal)

Alternative hypotheses:

Two-sided: $H_a: \sigma_1^2 \neq \sigma_2^2$
One-sided (greater): $H_a: \sigma_1^2 > \sigma_2^2$
One-sided (less): $H_a: \sigma_1^2 < \sigma_2^2$

Test statistic:

F = \frac{s_1^2}{s_2^2}

where $s_1^2$ and $s_2^2$ are the sample variances.

Convention: Place the larger sample variance in the numerator so that $F \geq 1$ . This simplifies interpretation: if $F$ is much larger than 1, it suggests the numerator variance is larger than the denominator variance.

Degrees of freedom: $df_1 = n_1 - 1$ (numerator), $df_2 = n_2 - 1$ (denominator)

P-value calculation:

One-sided (greater): $p = P(F_{df_1, df_2} > F_{obs})$
Two-sided: Since the F-distribution is not symmetric, we compute $p = 2 \times \min(P(F > F_{obs}), P(F < F_{obs}))$

Worked Example: Manufacturing Process ComparisonLink Copied

A quality engineer wants to compare the consistency of two production lines making precision components. She randomly samples 20 components from Line A and 25 from Line B, measuring a critical dimension.

In[4]:

Code

import numpy as np
from scipy import stats

# Measurements from two production lines (in microns from target)
np.random.seed(42)
line_a = np.random.normal(0, 4.5, 20)  # True std = 4.5 microns
line_b = np.random.normal(0, 7.2, 25)  # True std = 7.2 microns

# Step 1: Calculate sample statistics
n1, n2 = len(line_a), len(line_b)
var1 = np.var(line_a, ddof=1)
var2 = np.var(line_b, ddof=1)
std1, std2 = np.sqrt(var1), np.sqrt(var2)

# Step 2: Arrange so larger variance is in numerator
if var1 >= var2:
    f_stat = var1 / var2
    df1, df2 = n1 - 1, n2 - 1
    larger, smaller = "A", "B"
else:
    f_stat = var2 / var1
    df1, df2 = n2 - 1, n1 - 1
    larger, smaller = "B", "A"

# Step 3: Calculate p-value (two-sided)
p_upper = stats.f.sf(f_stat, df1, df2)
p_value_two_sided = 2 * p_upper  # For two-sided test

# Step 4: Get critical values
f_crit_upper = stats.f.ppf(0.975, df1, df2)
f_crit_lower = stats.f.ppf(0.025, df1, df2)

Let's trace through the calculation:

Step 1: Sample Statistics

s_A^2 = \frac{\sum_{i=1}^{20}(x_i - \bar{x})^2}{19}, \quad s_B^2 = \frac{\sum_{i=1}^{25}(x_i - \bar{x})^2}{24}

Step 2: F-Statistic

We place the larger variance in the numerator:

F = \frac{s_{larger}^2}{s_{smaller}^2}

Step 3: P-Value

Under $H_0$ , this F-statistic follows an $F_{df_1, df_2}$ distribution. We find the probability of observing a ratio this extreme.

Out[5]:

Console

F-Test for Comparing Variances: Production Line Comparison
=================================================================

Sample statistics:
-----------------------------------------------------------------
Line A: n = 20, variance = 18.66, std = 4.32 microns
Line B: n = 25, variance = 44.27, std = 6.65 microns

F-test calculation:
-----------------------------------------------------------------
Larger variance (Line B): 44.27
Smaller variance (Line A): 18.66
F-statistic: F = 44.27 / 18.66 = 2.372
Degrees of freedom: df1 = 24, df2 = 19

P-value (two-sided): 0.0588

Critical values (alpha = 0.05, two-sided):
  Lower: 0.426
  Upper: 2.452

Decision (alpha = 0.05):
-----------------------------------------------------------------
Fail to reject H_0: No significant evidence of different variances

Out[6]:

Visualization

Side-by-side boxplots comparing variability of two production lines. — Visual comparison of variability between two production lines. The boxplots show that Line B has greater spread than Line A.

F-distribution with shaded rejection region and observed F value. — F-distribution with test result. The F-test quantifies whether the observed difference is statistically significant or could have occurred by chance.

Important CaveatsLink Copied

The F-test for comparing variances is highly sensitive to non-normality. If the underlying populations are not normally distributed, the F-test can give misleading results even with moderate sample sizes. The t-test is less sensitive to departures from normality.

For this reason, many statisticians prefer alternatives that are less sensitive to non-normality:

Levene's test: Uses absolute deviations from the mean or median and is less sensitive to non-normality
Brown-Forsythe test: A variant of Levene's test using the median
Bartlett's test: More powerful under normality but sensitive to non-normality

Practical Recommendation

When checking the equal variance assumption for a t-test:

Visual inspection (boxplots, standard deviation ratio) is often sufficient
If a formal test is needed, use Levene's test rather than the F-test
When in doubt about equal variances, simply use Welch's t-test, which doesn't assume equal variances

In[7]:

Code

from scipy import stats

# Comparing F-test vs Levene's test
np.random.seed(42)
group1 = np.random.normal(50, 5, 30)
group2 = np.random.normal(50, 10, 30)

# F-test
f_stat = np.var(group2, ddof=1) / np.var(group1, ddof=1)
f_pvalue = 2 * stats.f.sf(f_stat, 29, 29)

# Levene's test (more robust)
levene_stat, levene_pvalue = stats.levene(group1, group2)

print("Comparison of variance equality tests:")
print("-" * 50)
print(f"F-test:      F = {f_stat:.2f}, p = {f_pvalue:.4f}")
print(f"Levene's:    W = {levene_stat:.2f}, p = {levene_pvalue:.4f}")
print()
print("Both detect the variance difference, but Levene's test")
print("is preferred because it's robust to non-normality.")

Out[7]:

Console

Comparison of variance equality tests:
--------------------------------------------------
F-test:      F = 4.28, p = 0.0002
Levene's:    W = 14.64, p = 0.0003

Both detect the variance difference, but Levene's test
is preferred because it's robust to non-normality.