Ridge Regularization (L2 Regularization) › Formula

Michael BrenndoerferJune 6, 20251 min read
Machine Learning from ScratchAll chapters
Part 4 of 9Regression Models
Chapter 34Ridge Regularization (L2 Regularization) › Formula
Reading section 2 of 9 · Formula · 1 minSection 2 of 9 · 1 min

Book contents

Machine Learning from Scratch9 parts · 66 chaptersView the whole book →
Active chapter 34 of 66

Section 2 of 9 · 1 min read

Ridge Regularization (L2 Regularization) › FormulaOne section · Sections in this chapter2 / 9 ↓
Sections in this chapter9
Reading tools Definitions & inline links

Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.

Article links

Make inline references clickable

The Ridge regression objective function is:

min⁡β{SSE+λ∑j=1pβj2}\min_{\beta} \left\{ \text{SSE} + \lambda \sum_{j=1}^p \beta_j^2 \right\}

where:

  • SSE\text{SSE} = sum of squared errors (measures how well the model fits the data)
  • λ\lambda = regularization parameter (controls the strength of the penalty; larger λ\lambda means stronger regularization)
  • βj\beta_j = coefficient for feature jj (the parameter we're trying to estimate)
  • pp = number of features (excluding the intercept β0\beta_0)
  • ∑j=1pβj2\sum_{j=1}^p \beta_j^2 = L2 penalty term (sum of squared coefficients)