Variable Relationships › Multiple Linear Regression: Multiple Predictors

Real-world phenomena typically depend on multiple factors rather than a single predictor. Multiple linear regression extends simple regression to model the relationship between a dependent variable and multiple independent variables simultaneously. This approach allows us to estimate the effect of each predictor while controlling for the others.

The multiple linear regression model is:

$$Y = \beta_0 + \beta_1 X_1 + \beta_2 X_2 + \cdots + \beta_p X_p + \epsilon$$

Where:

• $Y$: Dependent variable
• $X_1, X_2, \ldots, X_p$: Independent variables (predictors)
• $\beta_0$: Intercept
• $\beta_1, \beta_2, \ldots, \beta_p$: Regression coefficients
• $\epsilon$: Error term

Each coefficient $\beta_j$ represents the expected change in $Y$ for a one-unit change in $X_j$, holding all other predictors constant. This "holding constant" interpretation is important: it means we're estimating the partial effect of each variable, adjusting for the influence of other variables in the model. This capability makes multiple regression particularly useful for disentangling the effects of correlated predictors.

In matrix notation, the model can be written compactly as $\mathbf{Y} = \mathbf{X}\boldsymbol{\beta} + \boldsymbol{\epsilon}$, and the OLS solution is $\hat{\boldsymbol{\beta}} = (\mathbf{X}^T\mathbf{X})^{-1}\mathbf{X}^T\mathbf{Y}$. This formulation reveals the mathematical elegance of regression and enables efficient computational implementation.

While multiple regression with many predictors is difficult to visualize fully, we can illustrate the concept with two predictors. The regression model fits a plane through three-dimensional space, predicting Y from both X₁ and X₂.