The T-Test › Assumptions and Violations

Michael BrenndoerferJanuary 3, 20264 min read
Machine Learning from ScratchAll chapters
Part 2 of 9Hypothesis Testing
Chapter 15The T-Test › Assumptions and Violations
Reading section 7 of 11 · Assumptions and Violations · 4 minSection 7 of 11 · 4 min

Book contents

Machine Learning from Scratch9 parts · 66 chaptersView the whole book →
Active chapter 15 of 66

Section 7 of 11 · 4 min read

The T-Test › Assumptions and ViolationsOne section · Sections in this chapter7 / 11 ↓
Sections in this chapter11
Reading tools Definitions & inline links

Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.

Article links

Make inline references clickable

All t-tests rely on assumptions. Proper application requires understanding these assumptions, how to check them, and how violations affect the tests.

Core Assumptions

1. Independence

  • One-sample: Observations must be independent of each other
  • Two-sample (independent): Observations between groups must be independent; observations within groups must be independent
  • Paired: The pairs must be independent of each other (though observations within a pair are dependent by design)

Independence is often the most critical and hardest to verify assumption. It depends on study design rather than data characteristics.

2. Normality

The sampling distribution of the mean should be approximately normal. This is satisfied when:

  • The population is normally distributed, OR
  • The sample size is large enough for the Central Limit Theorem to apply

Tolerance of non-normality: Departures from normality usually have little effect on the t-test for:

  • Two-sided tests
  • Larger sample sizes (n > 30 per group)
  • Symmetric distributions

Departures matter more for:

  • Small samples from highly skewed distributions
  • One-sided tests
  • Comparing variances or making precise probability statements

3. Homogeneity of Variance (pooled t-test only)

The pooled two-sample t-test assumes equal population variances. Violations affect:

  • Type I error rate (can be inflated or deflated)
  • The bias is worse with unequal sample sizes

Solution: Use Welch's t-test, which doesn't assume equal variances.

Checking Assumptions

Out[15]:
Visualization
Q-Q plot showing sample quantiles versus theoretical normal quantiles.
Q-Q plot for checking normality. Points should fall approximately along the diagonal line if data are normally distributed. Deviations at the tails indicate skewness or heavy tails.
Side-by-side boxplots comparing variance between two groups.
Boxplot comparison for checking variance equality. Similar spreads suggest equal variances; markedly different spreads suggest using Welch's t-test.
In[16]:
Code
from scipy import stats

# Formal tests for assumptions
np.random.seed(42)
sample = np.random.normal(50, 10, 30)
group1 = np.random.normal(50, 8, 25)
group2 = np.random.normal(55, 15, 30)

# Shapiro-Wilk test for normality
stat, p = stats.shapiro(sample)
print("Normality check (Shapiro-Wilk test):")
print(f"  W = {stat:.4f}, p = {p:.4f}")
print(
    f"  {'No evidence against normality' if p > 0.05 else 'Evidence of non-normality'}"
)
print()

# Levene's test for equal variances
stat, p = stats.levene(group1, group2)
print("Variance equality check (Levene's test):")
print(f"  W = {stat:.4f}, p = {p:.4f}")
print(
    f"  {'No evidence of unequal variances' if p > 0.05 else 'Evidence of unequal variances'}"
)
Out[16]:
Console
Normality check (Shapiro-Wilk test):
  W = 0.9751, p = 0.6868
  No evidence against normality

Variance equality check (Levene's test):
  W = 7.3540, p = 0.0090
  Evidence of unequal variances