Gauss-Markov Assumptions › Mathematical Foundation

Michael BrenndoerferMay 22, 20251 min read
Machine Learning from ScratchAll chapters
Part 3 of 9Foundations
Chapter 27Gauss-Markov Assumptions › Mathematical Foundation
Reading section 4 of 11 · Mathematical Foundation · 1 minSection 4 of 11 · 1 min

Book contents

Section 4 of 11 · 1 min read

Gauss-Markov Assumptions › Mathematical FoundationOne section · Sections in this chapter4 / 11 ↓
Reading tools Definitions & inline links

Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.

Article links

Make inline references clickable

The Gauss-Markov theorem can be stated mathematically as follows:

Theorem: Under assumptions 1-5, the OLS estimator β^\hat{\beta} is the best linear unbiased estimator (BLUE) of β\beta.

This means:

  • Unbiased: E[β^]=βE[\hat{\beta}] = \beta
  • Efficient: Var(β^)\text{Var}(\hat{\beta}) is the smallest among all linear unbiased estimators
  • Consistent: As sample size increases, β^\hat{\beta} converges to the true β\beta

The OLS estimator is given by:

β^=(X′X)−1X′y\hat{\beta} = (X'X)^{-1}X'y

Where:

  • XX is the matrix of independent variables
  • yy is the vector of dependent variable
  • β^\hat{\beta} is the vector of estimated coefficients