Regression Analysis for Financial Relationships › Diagnosing Regression Problems

The validity of OLS inference depends on several assumptions about the error term. With financial time series data, these assumptions are frequently violated, leading to incorrect standard errors and misleading hypothesis tests. Understanding these assumptions and knowing how to detect their violations is essential for producing reliable results. Failing to address these issues can lead to false confidence in spurious relationships or obscure effects that are present.

The Gauss-Markov assumptions for OLS include:

• Linearity: The relationship between $y$ and $x$ is linear
• Exogeneity: $E[\varepsilon_i | x_i] = 0$ (errors are uncorrelated with explanatory variables)
• Homoskedasticity: $\text{Var}(\varepsilon_i) = \sigma^2$ (constant error variance)
• No autocorrelation: $\text{Cov}(\varepsilon_i, \varepsilon_j) = 0$ for $i \neq j$ (errors are uncorrelated across observations)
• Normality: $\varepsilon_i \sim N(0, \sigma^2)$ (required for exact inference in small samples)

When these assumptions hold, OLS is the Best Linear Unbiased Estimator (BLUE); no other linear unbiased estimator has smaller variance. This optimality result justifies the widespread use of OLS. However, when assumptions are violated, the BLUE property fails, and either the estimates themselves become biased or the standard errors become unreliable.

Heteroskedasticity occurs when the error variance changes across observations. In finance, this is common: volatility varies over time, as we explored in Chapter 18 on GARCH models. Periods of market stress have higher return volatility than calm periods. The variance of returns during the 2008 financial crisis was many times higher than during the calm period of 2005-2006. This time-varying volatility directly translates into heteroskedastic errors in regression models.

Heteroskedasticity doesn't bias the coefficient estimates, but it makes the standard errors incorrect, invalidating hypothesis tests. The OLS formula for standard errors assumes constant variance, so when variance varies, these standard errors are wrong. They might be too small (leading to false positives) or too large (leading to false negatives), depending on the pattern of heteroskedasticity.

Autocorrelation occurs when errors are correlated across time periods. This is particularly problematic in financial time series because returns often exhibit short-term momentum or mean-reversion patterns in their volatility. When today's error is correlated with yesterday's error, the observations are not truly independent. This reduces the effective sample size: 500 correlated observations contain less information than 500 independent observations.

The Durbin-Watson statistic tests for first-order autocorrelation:

$$DW = \frac{\sum_{t=2}^{n}(\hat{\varepsilon}_t - \hat{\varepsilon}_{t-1})^2}{\sum_{t=1}^{n}\hat{\varepsilon}_t^2}$$

where:

• $DW$: Durbin-Watson statistic
• $\hat{\varepsilon}_t$: residual at time $t$
• $n$: number of observations

The numerator sums squared differences between adjacent residuals. If errors are positively correlated, $\hat{\varepsilon}_t \approx \hat{\varepsilon}_{t-1}$, making the differences small and $DW$ low (closer to 0). If errors are uncorrelated, the differences are as large as the errors themselves, and the expected value is 2. If errors are negatively correlated, consecutive residuals tend to have opposite signs, making the differences large and $DW$ high (closer to 4).

Values near 2 indicate no autocorrelation, values below 2 indicate positive autocorrelation, and values above 2 indicate negative autocorrelation.

Multicollinearity arises when explanatory variables are highly correlated with each other. While not strictly a violation of OLS assumptions, it inflates standard errors and makes coefficient estimates unstable. The fundamental problem is identification: when two variables move together, it becomes difficult to determine which one is driving the effect. Mathematically, the matrix $\mathbf{X}'\mathbf{X}$ approaches singularity, making its inverse unstable.

The Variance Inflation Factor (VIF) quantifies multicollinearity:

$$VIF_j = \frac{1}{1 - R_j^2}$$

where:

• $VIF_j$: Variance Inflation Factor for variable $j$
• $R_j^2$: R-squared from regressing variable $j$ on all other explanatory variables

If variable $j$ is highly correlated with other predictors, $R_j^2$ approaches 1, causing the denominator to approach zero and $VIF_j$ to become very large. This mathematically quantifies how much the variance of the estimated coefficient is inflated due to collinearity. A VIF of 5 means the variance is five times what it would be if the variable were uncorrelated with other predictors, making the standard error about 2.2 times larger.

A VIF above 5 or 10 suggests problematic multicollinearity.