Multiple Linear Regression › Model Diagnostics: Residual Analysis

Michael BrenndoerferApril 16, 20253 min read
Machine Learning from ScratchAll chapters
Part 4 of 9Regression Models
Chapter 31Multiple Linear Regression › Model Diagnostics: Residual Analysis
Reading section 8 of 11 · Model Diagnostics: Residual Analysis · 3 minSection 8 of 11 · 3 min

Book contents

Machine Learning from Scratch9 parts · 66 chaptersView the whole book →
Active chapter 31 of 66

Section 8 of 11 · 3 min read

Multiple Linear Regression › Model Diagnostics: Residual AnalysisOne section · Sections in this chapter8 / 11 ↓
Sections in this chapter11
Reading tools Definitions & inline links

Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.

Article links

Make inline references clickable

After fitting a multiple linear regression model, examine the residuals to validate the model assumptions and identify potential problems. These diagnostic plots help ensure your model is appropriate and reliable.

Out[16]:
Visualization
Scatter plot of all 100 residuals against fitted values with a dashed zero-residual reference line.
Residuals vs Fitted Values plot for assessing linearity and homoscedasticity assumptions. The residuals (observed minus predicted values) are plotted against the fitted values to check for systematic patterns. A well-fitting model should show residuals randomly scattered around zero without clear trends or patterns. The red dashed line at y=0 represents perfect prediction. Systematic patterns like curves, funnels, or trends indicate violations of the linearity assumption or heteroscedasticity (non-constant variance), which can affect the reliability of coefficient estimates and statistical inference.
Q-Q plot of all 100 ordered residual values against theoretical normal quantiles with the fitted reference line.
Q-Q plot for assessing the normality assumption of residuals. Each point represents a residual's quantile compared to the corresponding normal distribution quantile. Points following the diagonal red line indicate good normality, while systematic deviations suggest non-normal residuals. This plot is more sensitive than histograms for detecting subtle departures from normality, particularly in the tails of the distribution where extreme values can significantly impact model performance and confidence intervals.
Scale-location plot of the square root of absolute residuals against fitted values with a dashed horizontal mean reference line.
Scale-Location plot for examining homoscedasticity (constant variance) assumption. The square root of absolute standardized residuals is plotted against fitted values to assess whether residual variance remains constant across the range of predictions. A horizontal red line indicates constant variance (homoscedasticity), while systematic patterns like increasing or decreasing spread suggest heteroscedasticity. This diagnostic is crucial for validating the assumption that residual variance doesn't depend on the level of the response variable.

These diagnostic plots are essential for validating your multiple linear regression model. The residuals vs fitted plot checks for linearity and constant variance, the Q-Q plot assesses normality, and the scale-location plot examines homoscedasticity. Together, they help ensure your model meets the necessary assumptions for reliable statistical inference.