The F-Test and F-Distribution › The F-Distribution: Where It Comes From

The F-distribution arises naturally when comparing two independent estimates of variance. To understand why, we need to trace through the mathematical foundations.

Recall from earlier chapters that when you estimate variance from a sample of $n$ observations drawn from a normal population with true variance $\sigma^2$, the quantity:

$$\frac{(n-1)s^2}{\sigma^2} \sim \chi^2_{n-1}$$

follows a chi-squared distribution with $n-1$ degrees of freedom. This result reflects the fact that $s^2$ is computed from $n$ observations but loses one degree of freedom because we estimate the mean from the same data.

Now suppose we have two independent samples from populations with variances $\sigma_1^2$ and $\sigma_2^2$:

• Sample 1: $n_1$ observations, sample variance $s_1^2$
• Sample 2: $n_2$ observations, sample variance $s_2^2$

Each sample variance, properly scaled, follows a chi-squared distribution:

$$\frac{(n_1-1)s_1^2}{\sigma_1^2} \sim \chi^2_{n_1-1} \quad \text{and} \quad \frac{(n_2-1)s_2^2}{\sigma_2^2} \sim \chi^2_{n_2-1}$$

The F-distribution is defined as the ratio of two independent chi-squared random variables, each divided by its degrees of freedom:

$$F = \frac{\chi_1^2 / df_1}{\chi_2^2 / df_2}$$

Substituting our variance estimates:

$$F = \frac{s_1^2 / \sigma_1^2}{s_2^2 / \sigma_2^2} = \frac{s_1^2}{s_2^2} \cdot \frac{\sigma_2^2}{\sigma_1^2}$$

Under the null hypothesis that $\sigma_1^2 = \sigma_2^2$, the ratio $\sigma_2^2 / \sigma_1^2 = 1$, which gives us:

$$F = \frac{s_1^2}{s_2^2} \sim F_{df_1, df_2}$$

where $df_1 = n_1 - 1$ and $df_2 = n_2 - 1$.

The unknown common variance cancels out! This is why the F-test works: we can test whether two population variances are equal without knowing their values. We only need the ratio of the sample variances, and we know its distribution under the null hypothesis.

The F-distribution has distinctive characteristics that set it apart from the normal and t-distributions:

Two degrees of freedom parameters: The F-distribution requires both $df_1$ (numerator) and $df_2$ (denominator). The order matters: $F_{5,10}$ is a different distribution from $F_{10,5}$. This asymmetry reflects the fact that we're comparing a specific numerator variance to a specific denominator variance.

Always non-negative: Since F is a ratio of variances (squared quantities), it cannot be negative. The distribution is defined only for $F \geq 0$.

Right-skewed: The F-distribution has a long right tail, especially when degrees of freedom are small. As both $df_1$ and $df_2$ increase, the distribution becomes more symmetric and concentrated.

Mean near 1 under the null: When comparing equal variances, F should be approximately 1. The expected value is $E[F] = df_2/(df_2 - 2)$ for $df_2 > 2$, which approaches 1 as $df_2$ grows large.

Reciprocal relationship: If $F \sim F_{df_1, df_2}$, then $1/F \sim F_{df_2, df_1}$. This is useful for two-tailed tests and for ensuring the larger variance is in the numerator.