Simple Linear Regression › Residual Analysis and Model Diagnostics

Michael BrenndoerferApril 7, 20253 min read
Machine Learning from ScratchAll chapters
Part 4 of 9Regression Models
Chapter 29Simple Linear Regression › Residual Analysis and Model Diagnostics
Reading section 6 of 10 · Residual Analysis and Model Diagnostics · 3 minSection 6 of 10 · 3 min

Book contents

Machine Learning from Scratch9 parts · 66 chaptersView the whole book →
Active chapter 29 of 66

Section 6 of 10 · 3 min read

Simple Linear Regression › Residual Analysis and Model DiagnosticsOne section · Sections in this chapter6 / 10 ↓
Sections in this chapter10
Reading tools Definitions & inline links

Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.

Article links

Make inline references clickable

Proper model validation requires checking whether the linear regression assumptions are met. These diagnostic plots help identify violations of key assumptions: linearity, independence, homoscedasticity (constant variance), and normality of residuals. Understanding these diagnostics is important for determining whether linear regression is appropriate for your data and for interpreting the reliability of your results.

Out[7]:
Visualization
Residuals vs fitted values plot for linearity and homoscedasticity.
Residuals vs Fitted Values plot for assessing linearity and homoscedasticity assumptions. The residuals (observed minus predicted values) are plotted against the fitted values to check for systematic patterns. A well-fitting model should show residuals randomly scattered around zero without clear trends or patterns. The red dashed line at y=0 represents perfect prediction. Systematic patterns like curves, funnels, or trends indicate violations of the linearity assumption or heteroscedasticity (non-constant variance), which can affect the reliability of coefficient estimates and statistical inference.
Q-Q plot for normality assessment.
Q-Q plot for assessing the normality assumption of residuals. Each point represents a residual's quantile compared to the corresponding normal distribution quantile. Points following the diagonal red line indicate good normality, while systematic deviations suggest non-normal residuals. This plot is more sensitive than histograms for detecting subtle departures from normality, particularly in the tails of the distribution where extreme values can significantly impact model performance and confidence intervals.
Scale-location plot for homoscedasticity.
Scale-Location plot for examining homoscedasticity (constant variance) assumption. The square root of absolute standardized residuals is plotted against fitted values to assess whether residual variance remains constant across the range of predictions. A horizontal red line indicates constant variance (homoscedasticity), while systematic patterns like increasing or decreasing spread suggest heteroscedasticity. This diagnostic is crucial for validating the assumption that residual variance doesn't depend on the level of the response variable.

These diagnostic plots are important tools for model validation. The residuals vs fitted plot helps identify non-linear patterns and heteroscedasticity, the Q-Q plot assesses normality assumptions, and the scale-location plot provides additional insight into variance patterns. Systematic violations of these assumptions may require data transformations, alternative modeling approaches, or careful interpretation of results.