Random Forest › Visualizing Random Forest

Michael BrenndoerferJune 30, 20255 min read
Machine Learning from ScratchAll chapters
Part 5 of 9Tree-Based Models
Chapter 42Random Forest › Visualizing Random Forest
Reading section 3 of 9 · Visualizing Random Forest · 5 minSection 3 of 9 · 5 min

Book contents

Machine Learning from Scratch9 parts · 66 chaptersView the whole book →
Active chapter 42 of 66

Section 3 of 9 · 5 min read

Random Forest › Visualizing Random ForestOne section · Sections in this chapter3 / 9 ↓
Sections in this chapter9
Reading tools Definitions & inline links

Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.

Article links

Make inline references clickable

Let's create visualizations that demonstrate how Random Forest works and how it differs from individual decision trees.

Out[6]:
Visualization
Decision boundary plot showing overfitting from individual decision tree.
This shows how individual decision trees can overfit to the training data, creating complex, irregular decision boundaries that may not generalize well to new data. Each tree makes different decisions based on the specific patterns it learned from its bootstrap sample.
Decision boundary plot showing smoother boundary from Random Forest ensemble.
This demonstrates how Random Forest combines multiple trees to create a smoother, more robust decision boundary. The ensemble approach reduces overfitting and creates more stable predictions by averaging the decisions of many individual trees.

Now let's visualize how the number of trees affects the Random Forest performance:

Out[7]:
Visualization
Line plot showing error convergence as number of trees increases in Random Forest.
This plot shows how Random Forest performance improves as more trees are added to the ensemble. The training error (blue line) decreases as more trees are added, while the test error (red line) initially decreases and then stabilizes. The out-of-bag error (green line) gives a good estimate of the test error without needing a separate validation set.

Note on Out-of-Bag (OOB) Error: The OOB error calculation requires a sufficient number of trees to provide reliable estimates. With very few trees, some observations may not be "out-of-bag" for any tree, making OOB scores unavailable. This is why we start the visualization from 10 trees rather than 1, and why scikit-learn may issue warnings about insufficient OOB samples when using very small numbers of trees. The OOB error provides a useful estimate of generalization performance without needing a separate validation set, but it becomes more reliable as the number of trees increases.