Ridge Regularization (L2 Regularization) › Visualizing Ridge Regression

Michael BrenndoerferJune 6, 20253 min read
Machine Learning from ScratchAll chapters
Part 4 of 9Regression Models
Chapter 34Ridge Regularization (L2 Regularization) › Visualizing Ridge Regression
Reading section 4 of 9 · Visualizing Ridge Regression · 3 minSection 4 of 9 · 3 min

Book contents

Machine Learning from Scratch9 parts · 66 chaptersView the whole book →
Active chapter 34 of 66

Section 4 of 9 · 3 min read

Ridge Regularization (L2 Regularization) › Visualizing Ridge RegressionOne section · Sections in this chapter4 / 9 ↓
Sections in this chapter9
Reading tools Definitions & inline links

Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.

Article links

Make inline references clickable

Let's visualize Ridge regression by plotting the Ridge regression objective function for different values of λ\lambda.

Out[6]:
Visualization
Scatter plot of 100 observations with five fitted regression lines for lambda values 0, 0.1, 1, 10, and 100; a six-entry legend identifies the data and fits, while five callouts report beta one shrinking from 47.133 to 21.186.
Effect of regularization parameter λ on Ridge regression. As λ increases from 0 (OLS) to 100, the regression line becomes flatter and the coefficient β₁ shrinks toward zero. This demonstrates how Ridge regularization constrains model complexity to prevent overfitting, with larger λ values producing more conservative fits.

As we increase the regularization parameter λ from 0 to 100, we can see the regression line becoming flatter and the slope coefficient β₁ decreasing. When λ = 0 (red line), we get the standard OLS solution with the steepest slope. As λ increases, the penalty for large coefficients forces the model to use smaller coefficients, resulting in a more conservative fit that's less likely to overfit. The gray data points show the original dataset, and each colored line represents a different level of regularization. Notice how the coefficient values (shown in the text box) decrease as λ increases.