Volume 3 of 4 · PDF edition

Trees, Explainability, and Unsupervised Learning

Decision trees and the ensembles that dominate tabular machine learning (random forests through XGBoost, LightGBM, and CatBoost), plus model explainability (SHAP, LIME), dimensionality reduction (PCA, t-SNE, UMAP), and clustering from k-means to HDBSCAN.

One part of Machine Learning from Scratch. The complete book remains free to read online; this paid edition is the carefully typeset PDF and companion notebook package you keep.

Pages
503 pages
Price
$24 one-time
Chapters
16 chapters
Buy nowLearn more
Machine Learning from Scratch, Volume 3: Trees, Explainability, and Unsupervised Learning cover

Learning outcomes

What you will learn

  • Build decision trees and understand how random forests and boosted ensembles improve them.
  • Compare XGBoost, LightGBM, CatBoost, and isolation forests by objective and operating trade-offs.
  • Explain model behavior with SHAP and LIME and reduce dimensions with PCA, t-SNE, and UMAP.
  • Choose and evaluate k-means, hierarchical, density-based, and HDBSCAN clustering.

Audience and prerequisites

Who this volume is for

  • Practitioners working with tabular data, nonlinear models, or model explainability.
  • Data scientists who need a principled path from supervised ensembles to unsupervised learning.
  • Engineers and analysts evaluating clustering and dimensionality-reduction methods for real datasets.

Prerequisites: Assumes the model-evaluation fundamentals from Volumes 1 and 2: loss, overfitting, regularization, and cross-validation.

Free online preview

Start with “CART (Classification and Regression Trees)”

Decision trees with greedy splitting algorithms

Approximately 45 minutes

Read the chapter

Exact contents

16 chapters across 3 parts

Every chapter below is included in the PDF and companion code. Links open the corresponding chapter in the free online edition.

Tree-Based Models

  1. 01CART (Classification and Regression Trees)
  2. 02Random Forest
  3. 03Boosted Trees
  4. 04XGBoost
  5. 05LightGBM
  6. 06CatBoost
  7. 07Isolation Forest

Explainability and Dimensionality Reduction

  1. 01PCA (Principal Component Analysis)
  2. 02t-SNE (t-Distributed Stochastic Neighbor Embedding)
  3. 03UMAP (Uniform Manifold Approximation and Projection)
  4. 04SHAP (SHapley Additive exPlanations)
  5. 05LIME (Local Interpretable Model-agnostic Explanations)

Unsupervised Learning

  1. 01K-means Clustering
  2. 02Hierarchical Clustering
  3. 03DBSCAN (Density-Based Spatial Clustering)
  4. 04HDBSCAN (Hierarchical DBSCAN)

Volume 3 PDF + code

Own this 503-page edition

Get the PDF, the runnable companion notebooks, and every future update and erratum for this volume. The corresponding online chapters stay free to read.

  • Carefully typeset PDF
  • Companion code for every included chapter
  • Future updates and errata included
  • Online chapters remain free to read

Volume 3 of 4

$24

one-time

Version 2026.08.2 · 503 pages · secure checkout via Stripe

Delivered by email · Free updates included

Compare editions

This volume or the complete book?

Volume 3: Trees, Explainability, and Unsupervised Learning

503 pages focused on Tree-Based Models and Explainability and Dimensionality Reduction and Unsupervised Learning. Best when this is the subject you need now.

$24

The complete Machine Learning from Scratch book

All four volumes in one 1,717-page PDF, with the complete companion code. Buying three distinct volumes also unlocks the complete book automatically.

$68

Compare all PDF editions

Also in the series

Explore the other volumes

Volume 1 · 402 pages

Statistical Foundations and Hypothesis Testing

Statistical Foundations for Machine Learning · Hypothesis Testing

Explore Volume 1

Volume 2 · 461 pages

Linear and Generalized Linear Models

Model Fundamentals and Diagnostics · Linear and Generalized Linear Models

Explore Volume 2

Volume 4 · 408 pages

Forecasting and Optimization

Time Series Forecasting · Optimization

Explore Volume 4

Version history

Kept current, not frozen in time

Each PDF purchase includes future editions. When the book changes, the updated copy appears in My books at no extra cost.

Current release

Edition 2026.08.2

Editorial improvements.

Earlier releases5

  1. 2026.08.1

    Editorial refinements, restored plot typography, clearer logarithmic axes, and refreshed companion code.

  2. 2026.08.0

    Editorial improvements throughout the book, with clearer explanations, refined presentation, improved plots and visualizations, and refreshed companion code.

  3. 2026.07.2

    Improved companion notebooks and validation; removed the obsolete CLT appendix; updated the support contact.

  4. 2026.07.1

    Validation edition bump: fixed typos in the SSE chapter.

  5. 2026.07.0

    Initial public PDF and companion-code release.