Polynomial Regression › Visualizing Polynomial Regression

Michael BrenndoerferJune 9, 20253 min read
Machine Learning from ScratchAll chapters
Part 4 of 9Regression Models
Chapter 32Polynomial Regression › Visualizing Polynomial Regression
Reading section 3 of 8 · Visualizing Polynomial Regression · 3 minSection 3 of 8 · 3 min

Book contents

Machine Learning from Scratch9 parts · 66 chaptersView the whole book →
Active chapter 32 of 66

Section 3 of 8 · 3 min read

Polynomial Regression › Visualizing Polynomial RegressionOne section · Sections in this chapter3 / 8 ↓
Sections in this chapter8
Reading tools Definitions & inline links

Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.

Article links

Make inline references clickable

Let's explore how polynomial regression can capture different types of non-linear relationships through a visualization.

Out[4]:
Visualization
Degree 1 polynomial regression fit compared with noisy data and the true cubic function.
Degree 1 (linear) polynomial regression captures only the general trend and underfits the curvature in the noisy cubic data.
Degree 2 polynomial regression fit compared with noisy data and the true cubic function.
Degree 2 (quadratic) polynomial regression captures more curvature but still simplifies the true cubic relationship.
Out[5]:
Visualization
Degree 3 polynomial regression fit compared with noisy data and the true cubic function.
Degree 3 (cubic) polynomial regression closely matches the true cubic relationship.
Degree 5 polynomial regression fit compared with noisy data and the true cubic function.
Degree 5 polynomial regression adds unnecessary complexity and begins to overfit the noisy observations.

The visualization above shows how different polynomial degrees capture the underlying non-linear relationship. We can see that:

  • Degree 1 (Linear): Captures only the general trend but misses the curvature
  • Degree 2 (Quadratic): Begins to capture some curvature but still oversimplifies
  • Degree 3 (Cubic): Closely matches the true cubic relationship
  • Degree 5: Overfits the data, creating unnecessary complexity and wiggles

This illustrates the fundamental trade-off in polynomial regression: higher degrees can capture more complex relationships but risk overfitting to noise in the data.

Bias-Variance Trade-off

This demonstrates the classic bias-variance trade-off in machine learning. Lower-degree polynomials have high bias (underfitting) but low variance, while higher-degree polynomials have low bias but high variance (overfitting). The optimal degree typically lies somewhere in between, where the model captures the true underlying pattern without fitting to noise.