Confidence Intervals and Test Assumptions › The Remarkable Equivalence

Here's the key insight that connects confidence intervals to hypothesis testing: a 95% confidence interval contains exactly those parameter values that would NOT be rejected by a two-sided hypothesis test at the 0.05 significance level.

Let's prove this algebraically. Consider testing $H_0: \mu = \mu_0$ versus $H_1: \mu \neq \mu_0$ at significance level $\alpha = 0.05$.

We reject $H_0$ when the test statistic exceeds the critical value:

$$\left| \frac{\bar{x} - \mu_0}{SE} \right| > t_{0.975, n-1}$$

Rearranging this inequality:

$$\begin{aligned} \left| \frac{\bar{x} - \mu_0}{SE} \right| > t_{0.975} &\iff |\bar{x} - \mu_0| > t_{0.975} \cdot SE \\ &\iff \bar{x} - \mu_0 > t_{0.975} \cdot SE \quad \text{OR} \quad \bar{x} - \mu_0 < -t_{0.975} \cdot SE \\ &\iff \mu_0 < \bar{x} - t_{0.975} \cdot SE \quad \text{OR} \quad \mu_0 > \bar{x} + t_{0.975} \cdot SE \end{aligned}$$

The last line says: we reject $H_0$ when $\mu_0$ falls outside the interval $[\bar{x} - t_{0.975} \cdot SE, \bar{x} + t_{0.975} \cdot SE]$.

But this interval is exactly the 95% confidence interval! So:

• $\mu_0$ inside the CI $\Rightarrow$ hypothesis test fails to reject $H_0$
• $\mu_0$ outside the CI $\Rightarrow$ hypothesis test rejects $H_0$

The confidence interval is the set of all parameter values that would not be rejected as null hypotheses.

This equivalence has important practical implications:

1. Confidence intervals are more informative than p-values. A p-value tells you whether one specific null hypothesis is rejected. A confidence interval tells you which values would be rejected and which wouldn't. It's like the difference between asking "Is the speed limit 65 mph?" versus "What range of speed limits are consistent with my speedometer reading?"

2. You can read significance from a CI. If a 95% CI for a difference doesn't include zero, the difference is significant at the 0.05 level. If the CI for a ratio doesn't include 1, the ratio is significant. You don't need to compute a separate test.

3. CIs convey precision. A CI of [0.1, 0.3] and a CI of [-2, 5] might both exclude zero (both "significant"), but the first shows you know the effect is small and positive, while the second shows you barely know anything at all.