Gauss-Markov Assumptions › The Five Gauss-Markov Assumptions

The Gauss-Markov theorem states that under certain assumptions, the OLS estimator is the best linear unbiased estimator (BLUE). This means it has the smallest variance among all linear unbiased estimators. The five key assumptions are:

The first assumption stipulates that the model must be linear in its parameters, though not necessarily in the variables themselves. This means that each regression coefficient multiplies a variable, and the sum of these products (plus an intercept and the error term) forms the predicted value. The model must be written as:

$$y_i = \beta_0 + \beta_1 x_{i1} + \beta_2 x_{i2} + \cdots + \beta_k x_{ik} + \varepsilon_i$$

Here, $y_i$ is the dependent variable, $x_{ij}$ are independent variables, $\beta_j$ are coefficients to be estimated, and $\varepsilon_i$ is the error term. The model is allowed to include transformed variables (such as logarithms, polynomials, or interactions), as long as the equation remains linear in the β's. For example, $y = \beta_0 + \beta_1 x + \beta_2 x^2 + \varepsilon$ still satisfies this assumption. The implication is that OLS cannot, without modification, fit models where parameters appear in exponents, denominators, or other non-linear forms.

This assumption requires that every observation in your sample is selected randomly and independently from the population of interest. In practical terms, this means each unit in the population has an equal probability of being included in the sample. Random sampling helps the sample represent the wider population so that inferences drawn from it are valid. If random sampling is violated, estimates can be systematically biased, and standard errors may not accurately reflect sampling variability. Random sampling also implies independence: the value of one observation does not influence or provide information about another.

This assumption requires that none of the independent variables is an exact linear function of any combination of the others. In other words, there must be no perfect or exact linear relationships among the explanatory variables. If perfect multicollinearity is present, it becomes mathematically impossible to estimate separate effects for the correlated variables using OLS, because the model cannot distinguish their individual influences. Even strong (but not perfect) correlations, known as high multicollinearity, can inflate the variances of the coefficient estimates, making statistical inference less reliable.coefficients can change erratically in response to small changes in the data. Ensuring that variables are linearly independent is essential for meaningful, stable regression results.

This is one of the most critical assumptions for ensuring unbiasedness of the OLS estimates. It states that the error term has an expected value of zero given any values of the independent variables:

$$E[\varepsilon_i | X] = 0$$

In effect, this means there are no omitted variables, measurement errors, or functional misspecifications that systematically influence the error term. If this assumption is violated.if, for example, a relevant variable is left out and is correlated with the included regressors.then the OLS estimates will be biased and cannot be trusted for causal interpretation. Zero conditional mean also means that all the relevant information for predicting $y$ is contained in $X$; nothing is left in the error term that could be predicted by $X$.

This assumption posits that the variance of the error term is constant across all values of the independent variables:

$$\text{Var}(\varepsilon_i | X) = \sigma^2$$

This uniformity in error variance.called homoscedasticity.means that no matter what the values of $X$ are, the spread or "noise" around the regression line remains the same. When this assumption holds, the standard errors calculated for the coefficients are accurate and hypothesis tests (such as t-tests) can be trusted. When it is violated (a situation known as heteroscedasticity), the efficiency of the OLS estimator is compromised, and the standard errors can be biased, potentially leading to misleading statistical inferences. Detecting and correcting heteroscedasticity (for example, using robust standard errors or transforming variables) is an important part of regression diagnostics.