CART (Classification and Regression Trees) › Visualizing CART

Michael BrenndoerferJune 27, 20254 min read
Machine Learning from ScratchAll chapters
Part 5 of 9Tree-Based Models
Chapter 41CART (Classification and Regression Trees) › Visualizing CART
Reading section 4 of 9 · Visualizing CART · 4 minSection 4 of 9 · 4 min

Book contents

Machine Learning from Scratch9 parts · 66 chaptersView the whole book →
Active chapter 41 of 66

Section 4 of 9 · 4 min read

CART (Classification and Regression Trees) › Visualizing CARTOne section · Sections in this chapter4 / 9 ↓
Sections in this chapter9
Reading tools Definitions & inline links

Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.

Article links

Make inline references clickable

Let's create visualizations that demonstrate how CART builds decision trees and makes predictions. We'll show both the tree structure and the decision boundaries it creates.

Out[5]:
Visualization
Heat map visualization showing rectangular decision regions created by CART splits, with each color representing different class predictions based on feature thresholds.
CART Classification Decision Boundary: This visualization shows how CART creates rectangular decision regions through axis-parallel splits. Each colored region represents a different class prediction, with the boundaries formed by the tree''s splitting decisions. The scatter points show the actual training data.
Out[6]:
Visualization
Tree diagram showing CART decision tree structure with nodes displaying splitting criteria and impurity.
CART Tree Structure Visualization: This dendrogram shows the complete decision tree structure for the classification problem. Each node displays the splitting criterion (feature and threshold), the Gini impurity, the number of samples, and the class distribution. The color intensity indicates the purity of each node, with darker colors representing more pure (less impure) nodes. The tree shows how the algorithm recursively partitions the data based on feature thresholds.

The tree visualization shows the hierarchical decision-making process, with each node displaying the splitting criterion and the resulting class distribution.

Out[7]:
Visualization
Scatter plot showing training data and piecewise constant predictions from CART regression tree.
CART Regression Tree Predictions: This plot demonstrates how CART creates piecewise constant predictions for regression problems. The blue scatter points represent the training data, while the red step function shows the tree''s predictions. Each horizontal segment corresponds to a leaf node in the tree, with the prediction value being the mean of all training samples that fall into that leaf.
Out[8]:
Visualization
Vertical bar chart comparing two features by their importance scores, showing which variable contributes most to reducing impurity across the decision tree splits.
Feature Importance in CART: This bar chart shows the relative importance of each feature in the CART model, calculated based on how much each feature reduces impurity across all splits in the tree. Higher values indicate features that contribute more to the model''s decision-making process. This visualization helps identify which features are most influential for predictions.