Regression Analysis for Financial Relationships › Hypothesis Testing and Model Evaluation

Estimating coefficients is only the first step. We need to assess whether these estimates are statistically meaningful and how well the model explains variation in the dependent variable. The challenge is that our estimates are random variables: they depend on the particular sample we observed. Different samples would produce different estimates. Statistical inference quantifies this uncertainty, telling us how confident we can be in our conclusions.

The uncertainty in coefficient estimates comes from sampling variation. We observe only one realization of the data, but if we could repeatedly sample from the same underlying process, we would get different coefficient estimates each time. The distribution of these hypothetical estimates determines how confident we can be in any single estimate.

Under the classical OLS assumptions (which we'll examine shortly), the coefficient estimates follow a multivariate normal distribution:

where:

• $\hat{\boldsymbol{\beta}}$: estimator vector
• $\boldsymbol{\beta}$: true coefficient vector
• $\sigma^2$: variance of the error term
• $(\mathbf{X}'\mathbf{X})^{-1}$: inverse of the moment matrix of regressors

This result tells us that OLS estimates are unbiased: their expected value equals the true parameter. The covariance matrix $\sigma^2(\mathbf{X}'\mathbf{X})^{-1}$ determines the precision of each estimate. The diagonal elements give the variances of individual coefficients, while off-diagonal elements capture how estimation errors in different coefficients relate to each other.

Since we don't know $\sigma^2$, we estimate it using the residual variance:

where:

• $\hat{\sigma}^2$: estimated variance of the error term
• $\hat{\varepsilon}_i$: residual for observation $i$
• $SSR$: sum of squared residuals
• $n$: number of observations
• $k$: number of explanatory variables (excluding intercept)

The denominator $n - k - 1$ accounts for the degrees of freedom lost in estimating $k + 1$ parameters (including the intercept). This adjustment ensures the variance estimate is unbiased. With more parameters to estimate, we have less information available to estimate the error variance, so we divide by a smaller number.

The standard error of coefficient $j$ is:

where:

• $SE(\hat{\beta}_j)$: standard error of the $j$-th coefficient
• $\hat{\sigma}$: estimated standard deviation of the error term
• $[(\mathbf{X}'\mathbf{X})^{-1}]_{jj}$: the $j$-th diagonal element of the inverse matrix

The term $[(\mathbf{X}'\mathbf{X})^{-1}]_{jj}$ reflects both the variance of the $j$-th regressor and its correlation with other regressors. High variance in $x_j$ reduces the standard error (more information), while high correlation with other variables increases it (multicollinearity). Intuitively, if a variable moves a lot, we can more precisely estimate its effect. But if it moves together with other variables, it becomes harder to isolate its unique contribution.

To test whether a coefficient differs from zero, we compute the t-statistic:

where:

• $t_j$: t-statistic for the $j$-th coefficient
• $\hat{\beta}_j$: estimated coefficient
• $SE(\hat{\beta}_j)$: standard error of the estimate

This ratio compares the magnitude of the estimate to its precision. A large t-statistic (in absolute value) means the estimate is large relative to its uncertainty, suggesting a non-zero effect. Under the null hypothesis that $\beta_j = 0$, this statistic follows a t-distribution with $n - k - 1$ degrees of freedom. The t-distribution accounts for the additional uncertainty from estimating $\sigma^2$.

The p-value gives the probability of observing a t-statistic as extreme as or more extreme than the one calculated, assuming the null hypothesis is true. A small p-value (typically below 0.05 or 0.01) suggests the coefficient is statistically significantly different from zero. The logic is indirect: if the null hypothesis were true, the observed result would be very unlikely, so we reject the null.

In financial applications, statistical significance doesn't guarantee economic significance. A beta estimate might be highly significant (p < 0.001) but the relationship might be too weak or unstable to be useful for trading. Conversely, economically meaningful effects might not achieve statistical significance in small samples. The p-value depends on both the magnitude of the effect and the precision of our estimate, so a small but precisely estimated effect can be highly significant while a large but imprecisely estimated effect may not be.

The beta estimate is highly significant, with a t-statistic far exceeding critical values. This means we can confidently reject the null hypothesis that the stock has no market exposure. The alpha estimate, while positive, may or may not be statistically significant depending on the random realization. This illustrates the challenge of detecting true alpha.

The coefficient of determination, $R^2$, measures the proportion of variance in the dependent variable explained by the model. This statistic answers a fundamental question: how much of the variation in returns can we attribute to the factors in our model, versus unexplained idiosyncratic variation?

where:

• $R^2$: coefficient of determination
• $SSR$: sum of squared residuals (unexplained variation)
• $SST$: total sum of squares (total variation)
• $y_i$: observed value of dependent variable
• $\hat{y}_i$: fitted value for observation $i$
• $\bar{y}$: sample mean of the dependent variable

$R^2$ ranges from 0 to 1, with higher values indicating better fit. An $R^2$ of 0.5 means that 50% of the variation in returns is explained by the model, while 50% remains unexplained. The unexplained portion represents idiosyncratic risk and the effects of factors not included in the model.

For single-stock regressions against the market, $R^2$ typically ranges from 0.2 to 0.6 for individual stocks. Lower values indicate more idiosyncratic risk that cannot be diversified through market exposure alone. For portfolios, $R^2$ tends to be higher because idiosyncratic risks cancel out. A well-diversified portfolio might have $R^2$ of 0.9 or higher against the market, indicating that almost all its risk is systematic.

The F-test assesses whether the model as a whole has explanatory power, meaning whether at least one coefficient (excluding the intercept) is significantly different from zero. While t-tests evaluate individual coefficients, the F-test provides a joint test of all slope coefficients simultaneously. This distinction matters because individual coefficients might be insignificant due to multicollinearity even when the model as a whole explains significant variation.

where:

• $F$: F-statistic for the model
• $SST$: total sum of squares
• $SSR$: sum of squared residuals
• $R^2$: coefficient of determination
• $k$: number of slope coefficients
• $n$: number of observations

The F-statistic compares the variance explained by the model (per parameter) to the unexplained variance (per degree of freedom). A value significantly greater than 1 indicates the model captures more signal than expected by random chance. The numerator represents average explained variation per variable, while the denominator represents unexplained variation per observation. If the model has no explanatory power, these should be roughly equal, yielding an F-statistic near 1.

Under the null hypothesis that all slope coefficients equal zero, the F-statistic follows an F-distribution with $k$ and $n - k - 1$ degrees of freedom.