Spline Regression › Visualizing Spline Regression

Michael BrenndoerferJune 18, 20255 min read
Machine Learning from ScratchAll chapters
Part 4 of 9Regression Models
Chapter 33Spline Regression › Visualizing Spline Regression
Reading section 3 of 8 · Visualizing Spline Regression · 5 minSection 3 of 8 · 5 min

Book contents

Machine Learning from Scratch9 parts · 66 chaptersView the whole book →
Active chapter 33 of 66

Section 3 of 8 · 5 min read

Spline Regression › Visualizing Spline RegressionOne section · Sections in this chapter3 / 8 ↓
Sections in this chapter8
Reading tools Definitions & inline links

Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.

Article links

Make inline references clickable

Let's explore how spline regression works by visualizing the basis functions and comparing different spline fits to polynomial regression.

Out[8]:
Visualization
Scatter plot showing complex non-linear data points with underlying true function curve.
Original data with complex non-linear relationship and the true underlying function. This synthetic dataset demonstrates the type of data where spline regression excels over simple linear or polynomial approaches.
Multiple smooth curves showing B-spline basis functions with compact support over different intervals.
B-spline basis functions showing how smooth, compactly supported functions form the building blocks of spline models. Each basis function is non-zero only over a limited range and smoothly transitions between segments.
Out[9]:
Visualization
Comparison plot showing polynomial regression with artifacts versus smooth spline fits with different knot counts.
Comparison of different regression methods: polynomial regression (degree 5) versus spline regression with 3 and 7 knots. Spline regression captures the underlying pattern more effectively with fewer artifacts.
Residual plots comparing random error distribution of splines versus systematic patterns in polynomial regression.
Residuals comparison showing how spline regression provides better fit quality with more random error distribution compared to polynomial regression, which shows systematic patterns.

The visualization above shows several key aspects of spline regression. The first plot shows our synthetic data with a complex non-linear relationship. The second plot shows the B-spline basis functions, which are smooth, compactly supported functions that form the building blocks of our spline model. Notice how each basis function is non-zero only over a limited range, and they smoothly transition between segments.

The third plot compares different regression approaches, showing how spline regression can capture the underlying pattern more effectively than polynomial regression, especially with an appropriate number of knots. The fourth plot shows the residuals, revealing how spline regression can provide better fit quality with fewer artifacts compared to high-degree polynomials.

Knot Selection Impact

You can see the importance of knot selection. Too few knots (3) may underfit the data, while too many knots (7) might overfit. The optimal number of knots typically requires cross-validation to determine, balancing model complexity with generalization performance.