Machine Learning from Scratch
Learn the math and intuition behind machine learning algorithms, then implement them in Python
The book spans 4 volumes. Read in order, or browse the contents to find the topic you need.
1,717 pages across 66 chapters
Browse contentsRead Machine Learning from Scratch free online
Machine Learning from Scratch is for readers who want to see how algorithms actually work. We derive each method from the math, build the intuition step by step, and implement it in Python so you can inspect every decision.
Explore the book’s 4 volumes
Machine Learning from Scratch is organised into 4 volumes. Each volume covers a subject, divided into parts and chapters. Read from the beginning, or choose the volume you need.
66 published chapters · Free to read online
Volume 1Statistical Foundations and Hypothesis Testing2 parts · 22 chapters, including planned additionsBuy this volumeBuy PDF · $24
Part I
Statistical Foundations for Machine Learning
11 chapters01Types of DataComplete guide to data classification - quantitative, qualitative, discrete & continuous19 min02Descriptive StatisticsComplete guide to summarizing and understanding data with measures of central tendency, variability, and distribution shape26 min03Probability BasicsFoundation of statistical reasoning covering random variables, probability distributions, expected value, variance, and conditional probability46 min04Central Limit TheoremFoundation of statistical inference covering convergence behavior, sample size requirements, and practical applications in data science12 min05Data SamplingComplete guide to sampling theory and methods covering simple random sampling, stratified sampling, cluster sampling, sampling error, and uncertainty quantification16 min06Variable RelationshipsComplete guide to covariance, correlation, and regression analysis covering how to measure, model, and interpret variable associations26 min07Probability DistributionsComplete guide to normal, t-distribution, binomial, Poisson, exponential, and log-normal distributions with practical applications18 min08Data VisualizationComplete guide to histograms, box plots, and scatter plots for exploratory data analysis17 min09Data QualityComplete guide to data quality and outliers covering measurement error, bias, missing data, and imputation34 min10Statistical InferenceComplete guide to drawing conclusions from data covering point and interval estimation, confidence intervals, hypothesis testing, and p-values25 min11Statistical ModellingComplete guide to building and evaluating predictive models covering model fit metrics, bias-variance tradeoff, and cross-validation19 min
Part II
Hypothesis Testing
11 chapters12P-values and Hypothesis Test SetupFoundation of hypothesis testing covering p-values, null and alternative hypotheses, one-sided vs two-sided tests, and test statistics21 min13Confidence Intervals and Test AssumptionsMathematical equivalence between confidence intervals and hypothesis tests, test assumptions, and choosing between z and t tests23 min14The Z-TestComplete guide to z-tests including one-sample, two-sample, and proportion tests22 min15The T-TestComplete guide to t-tests including one-sample, two-sample (pooled and Welch), paired tests, assumptions, and decision framework29 min16The F-Test and F-DistributionF-distribution, F-test for comparing variances, F-test in regression, and nested model comparison24 min17ANOVA (Analysis of Variance)One-way ANOVA, post-hoc tests, assumptions, and when to use ANOVA23 min18Type I and Type II ErrorsUnderstanding false positives, false negatives, statistical power, and the tradeoff between error types32 min19Sample Size, Minimum Detectable Effect, and PowerPower analysis, sample size determination, MDE calculation, and avoiding underpowered studies29 min20Effect Sizes and Statistical SignificanceCohen's d, practical significance, interpreting effect sizes, and why tiny p-values can mean tiny effects20 min21Multiple ComparisonsFamily-wise error rate, false discovery rate, Bonferroni correction, Holm's method, and Benjamini-Hochberg procedure29 min22Summary and Practical Guide to Hypothesis TestingPractical reporting guidelines, summary of key concepts, test selection parameters table, multiple comparison corrections table, and scipy.stats functions reference18 min
Volume 2Linear and Generalized Linear Models4 parts · 30 chapters, including planned additionsBuy this volumeBuy PDF · $24
Part III
Model Fundamentals and Diagnostics
8 chapters23Sum of Squared Errors (SSE)The fundamental metric for measuring regression model performance and prediction accuracy19 min24R-squaredUnderstanding the coefficient of determination and model fit metrics7 min25StandardizationNormalizing features for fair comparison in machine learning algorithms12 min26NormalizationMin-max scaling to transform features to a common [0, 1] range for neural networks and distance-based algorithms14 min27Gauss-Markov AssumptionsFoundation of linear regression and OLS estimation covering linearity, independence, homoscedasticity, normality, and practical testing methods16 min28MulticollinearityUnderstanding the impact of multicollinearity on regression models41 min
29Maximum Likelihood EstimationDerive estimators from likelihood, connect Gaussian noise to squared error, and inspect identifiability and uncertainty.Coming soon
30Gradient Descent and Numerical OptimizationBuild gradient descent, stochastic updates, and Newton steps while examining scaling, convergence, and numerical stability.Coming soon
Part IV
Model Evaluation and Selection
3 chapters31Validation Strategies and Data LeakageDesign train, validation, and test partitions that respect preprocessing, repeated entities, and temporal information.Coming soon
32Classification Metrics and Decision ThresholdsConnect confusion matrices, precision-recall, ROC curves, class imbalance, and decision costs to threshold selection.Coming soon
33Hyperparameter Search and Nested Cross-ValidationCompare grid and random search and estimate the generalization of a complete model-selection procedure with nested validation.Coming soon
Part V
Linear and Generalized Linear Models
14 chapters34Simple Linear RegressionMathematical foundations, formulas, and step-by-step implementation48 min35Ordinary Least Squares (OLS)Vector notation and matrix operations for regression33 min36Multiple Linear RegressionExtending linear regression to multiple predictors41 min37Polynomial RegressionModeling non-linear relationships with polynomial features38 min38Spline RegressionFlexible non-parametric regression using piecewise polynomials65 min
39Generalized Additive ModelsExtend spline regression to additive predictors with smoothness penalties, link functions, and interpretable component effects.Coming soon
40Ridge Regularization (L2 Regularization)L2 penalty for handling multicollinearity and overfitting36 min41Lasso Regularization (L1 Regularization)L1 penalty for feature selection and overfitting prevention60 min42Elastic Net RegularizationCombining L1 and L2 penalties for optimal regularization52 min43Logistic RegressionBinary classification using the logistic function46 min44Multinomial Logistic RegressionMulti-class classification extension of logistic regression48 min45Poisson RegressionModeling count data with Poisson distribution49 min46Generalized Linear ModelsExtending linear regression to non-normal distributions55 min47Quantile RegressionEstimate conditional quantiles with pinball loss and diagnose crossing quantiles, heterogeneous spread, and interval coverage.Coming soon
Part VI
Survival Analysis
5 chapters48Time-to-Event Data, Censoring, and Hazard FunctionsRepresent event times and censoring, derive survival-hazard relationships, and identify assumptions behind time-to-event estimation.Coming soon
49Kaplan–Meier and Nelson–Aalen EstimationBuild risk sets and estimate survival and cumulative hazard nonparametrically with censoring-aware uncertainty.Coming soon
50Cox Proportional HazardsDerive Cox partial likelihood, interpret hazard ratios, estimate baseline survival, and diagnose proportional-hazards assumptions.Coming soon
51Accelerated Failure-Time ModelsModel log event times, derive censored likelihoods, interpret time ratios, and compare parametric survival assumptions.Coming soon
52Evaluating Survival PredictionsEvaluate risk rankings and survival probabilities with censoring-aware concordance, time-dependent metrics, and calibration.Coming soon
Volume 3Trees, Explainability, and Unsupervised Learning6 parts · 25 chapters, including planned additionsBuy this volumeBuy PDF · $24
Part VII
Supervised Learning Beyond Linear Models
5 chapters53k-Nearest NeighborsBuild distance-based classification and regression and examine neighborhood size, feature scaling, and dimensionality.Coming soon
54Naive BayesDerive probabilistic classification with conditional independence, Gaussian and multinomial likelihoods, and smoothing.Coming soon
55Linear and Quadratic Discriminant AnalysisDerive Gaussian discriminant rules and compare shared, class-specific, and regularized covariance assumptions.Coming soon
56Support Vector Machines and Support Vector RegressionDevelop maximum-margin classification, soft-margin objectives, kernels, and epsilon-insensitive regression.Coming soon
57Multilayer Perceptrons and BackpropagationBuild a small feedforward network and derive the chain-rule gradients needed to train it from scratch.Coming soon
Part VIII
Tree-Based Models
7 chapters58CART (Classification and Regression Trees)Decision trees with greedy splitting algorithms45 min59Random ForestEnsemble method combining multiple decision trees with bagging43 min
60Random Survival ForestsExtend tree ensembles to censored event times using survival splitting, terminal-node estimators, and survival-specific evaluation.Coming soon
61Boosted TreesGradient boosting for improved predictive performance44 min62XGBoostOptimized gradient boosting with advanced regularization techniques78 min63LightGBMFast gradient boosting with leaf-wise tree growth52 min64CatBoostGradient boosting with categorical feature handling42 minPart IX
Unsupervised Learning
6 chapters65K-means ClusteringPartitioning data into k clusters using centroid-based approach91 min
66Gaussian Mixture Models and Expectation-MaximizationMove from hard k-means assignments to probabilistic mixtures, latent responsibilities, and EM likelihood optimization.Coming soon
67Hierarchical ClusteringTree-based clustering with agglomerative or divisive methods51 min68DBSCAN (Density-Based Spatial Clustering)Density-based clustering for arbitrary shaped clusters and noise detection58 min69HDBSCAN (Hierarchical DBSCAN)Hierarchical density-based clustering with varying density37 min70Isolation ForestUnsupervised anomaly detection with random trees and path lengths48 minPart X
Dimensionality Reduction
3 chapters71PCA (Principal Component Analysis)Dimensionality reduction and feature extraction23 min72t-SNE (t-Distributed Stochastic Neighbor Embedding)Non-linear visualization technique37 min73UMAP (Uniform Manifold Approximation and Projection)Non-linear dimensionality reduction preserving local and global structure40 min
Part XI
Model Interpretation
2 chaptersPart XII
Calibration and Prediction Uncertainty
2 chapters76Probability CalibrationMeasure and improve probability reliability while separating calibration, discrimination, and decision quality.Coming soon
77Prediction Intervals and Split Conformal PredictionDistinguish confidence and prediction intervals and build split conformal intervals and classification sets with explicit coverage assumptions.Coming soon
Volume 4Forecasting and Optimization3 parts · 15 chapters, including planned additionsBuy this volumeBuy PDF · $24
Part XIII
Time Series Forecasting
6 chapters78Forecasting Baselines, Temporal Validation, and BacktestingEstablish naive and seasonal baselines and evaluate forecasts with rolling origins, explicit horizons, and leakage-free feature availability.Coming soon
79ETS (Exponential Smoothing)Classical time series forecasting with trend and seasonality61 min80SARIMA (Seasonal ARIMA)Autoregressive integrated moving average with seasonal components35 min81ProphetFacebook's forecasting tool for business time series with holidays43 min82N-BEATSNeural basis expansion analysis for interpretable time series forecasting51 min83N-HiTSNeural hierarchical interpolation for time series forecasting71 minPart XIV
Constrained Optimization
6 chapters84Objectives, Decision Variables, Constraints, and FeasibilityTranslate operational decisions into objective functions and feasible sets and distinguish formulation, optimization, and solution validation.Coming soon
85MILP FactoryMixed integer linear programming for production planning67 min86Min Cost Flow SlottingNetwork flow optimization for resource allocation72 min87QP PortfolioQuadratic programming for portfolio optimization and risk management46 min88CP-SAT RosteringConstraint programming for employee scheduling and rostering56 min89VRPTW RoutingVehicle routing problem with time windows for logistics67 minPart XV
Heuristic and Evolutionary Optimization
3 chapters90Local Search and Simulated AnnealingSearch discrete neighborhoods with hill climbing, restarts, and annealing while tracking feasibility and evaluation budgets.Coming soon
91Genetic AlgorithmsBuild evolutionary search from representation, selection, crossover, mutation, and constraint handling, then compare it with simpler baselines.Coming soon
92Differential EvolutionDevelop continuous population search through vector differences, crossover, greedy replacement, and bound handling.Coming soon
More about the book
From the elegant simplicity of linear regression to the sophisticated power of gradient boosting and neural networks, every concept is built from first principles. You won't just learn how to use scikit-learn. You'll understand exactly what happens under the hood when you call fit() and predict(). Each algorithm is derived mathematically, explained intuitively, and implemented in clean, Python code.
Whether you're a data scientist looking to deepen your expertise, an ML engineer preparing for system design interviews, a researcher needing solid foundations, or a quantitative analyst applying machine learning to financial problems, this handbook provides the rigorous yet accessible treatment you need. Every chapter includes derivations you can follow, visualizations that build intuition, and implementations you can run and extend.
Covering statistics fundamentals, regression techniques, tree-based methods, clustering algorithms, dimensionality reduction, time series forecasting, and mathematical optimization, this is the definitive resource for anyone serious about mastering machine learning from the ground up.
Own the book
The whole book, beautifully typeset, yours forever.
Reading online is free, and always will be. The paid edition is the book you keep: a carefully typeset PDF of all 4 volumes, with the full companion code, and every future update included.
All ~1,717 pages as one beautifully typeset PDF
The complete companion code, every chapter as a runnable notebook
All future expansions, updates, and errata, free, forever
Downloadable editions for durable offline reading
EPUB edition included when it ships
Best value, save $28 (29%)
$68
one-time
instead of $96 for the 4 volumes bought separately
5.0 from 12 readers
Secure checkout via Stripe · Individual PDF links delivered to your inbox
Reader verdict
5.0
out of 5
12 five-star reader reviews
Genuine feedback, with names abbreviated for privacy.
Meet the community“I would like to thank you for your work. Specifically, the chapter: 'LIME Explainability: Complete Guide to Local Interpretable Model-Agnostic Explanations' was extremely helpful for me in understanding LIME and communicating to colleagues. Your work is inspiring and valuable. Thank you for doing it.”
“Really appreciate the way you have explained the concepts. Simple to understand, yet builds up knowledge exponentially, all while reinforcing it with examples repeatedly. I've read the last two chapters `74.Self-attention & 75.Q,K,V` and I've to admit, my concepts have never been clearer. You really are doing an amazing job breaking down maths and complex concepts like a story!”
“I am enjoying reading your writings. I appreciate the effort you put into it. I want to drop a note of thanks. Thank you!”
“I found your book Language AI Handbook's explanations of Transformer architectures and production deployment strategies to be exceptionally clear and insightful”
“Your writing feels like a treasure trove of great material. So, thank you for compiling everything on your website!”
“Thanks so much for establishing such an amazing community. I appreciate the opportunity to be a learner and a contributor to the community.”
“Thank you Michael. It's top!”
“I read a few of your blogs on NLP and oh WOW, they really are something, now I exactly know how all of the terms Entropy, Cross-entropy and how it is connected to Perplexity, will definitely read more of your blogs.”
“I came across your collection of online handbooks covering quantitative finance, AI, and machine learning. I wanted to reach out and thank you for making such high-quality material openly available. They are incredibly helpful for my CFA Level II review and for brushing up ahead of ML interviews.”
“I just came across your website where your books are available for free to read. Thank you so much for making this available to the public. This will help a lot of people.”
“Thanks for putting cool stuff out into the world.”
“I came across one of your books, Language AI Handbook, and man, I'm loving it. I've read many books but have never found such a comprehensive book on large models that covers everything from start to finish with such depth.”
Read all 12 reader reviews
“I would like to thank you for your work. Specifically, the chapter: 'LIME Explainability: Complete Guide to Local Interpretable Model-Agnostic Explanations' was extremely helpful for me in understanding LIME and communicating to colleagues. Your work is inspiring and valuable. Thank you for doing it.”
“Really appreciate the way you have explained the concepts. Simple to understand, yet builds up knowledge exponentially, all while reinforcing it with examples repeatedly. I've read the last two chapters `74.Self-attention & 75.Q,K,V` and I've to admit, my concepts have never been clearer. You really are doing an amazing job breaking down maths and complex concepts like a story!”
“Thanks so much for establishing such an amazing community. I appreciate the opportunity to be a learner and a contributor to the community.”
“I just came across your website where your books are available for free to read. Thank you so much for making this available to the public. This will help a lot of people.”
“I am enjoying reading your writings. I appreciate the effort you put into it. I want to drop a note of thanks. Thank you!”
“Thank you Michael. It's top!”
“Thanks for putting cool stuff out into the world.”
“I found your book Language AI Handbook's explanations of Transformer architectures and production deployment strategies to be exceptionally clear and insightful”
“Your writing feels like a treasure trove of great material. So, thank you for compiling everything on your website!”
“I came across your collection of online handbooks covering quantitative finance, AI, and machine learning. I wanted to reach out and thank you for making such high-quality material openly available. They are incredibly helpful for my CFA Level II review and for brushing up ahead of ML interviews.”
“I read a few of your blogs on NLP and oh WOW, they really are something, now I exactly know how all of the terms Entropy, Cross-entropy and how it is connected to Perplexity, will definitely read more of your blogs.”
“I came across one of your books, Language AI Handbook, and man, I'm loving it. I've read many books but have never found such a comprehensive book on large models that covers everything from start to finish with such depth.”
Author and edition details
About the author and this edition
Michael Brenndoerfer
Michael has spent more than a decade working across software engineering, data, AI, and business. He writes to understand difficult ideas more deeply and to share what he learns in a clear, practical way.
- Edition
- First edition
- Published
- Last reviewed
Version history
Kept current, not frozen in time
Each PDF purchase includes future editions. When the book changes, the updated copy appears in My books at no extra cost.
Current release
Edition 2026.08.2
Editorial improvements.
Earlier releases5
2026.08.1
Editorial refinements, restored plot typography, clearer logarithmic axes, and refreshed companion code.
2026.08.0
Editorial improvements throughout the book, with clearer explanations, refined presentation, improved plots and visualizations, and refreshed companion code.
2026.07.2
Improved companion notebooks and validation; removed the obsolete CLT appendix; updated the support contact.
2026.07.1
Validation edition bump: fixed typos in the SSE chapter.
2026.07.0
Initial public PDF and companion-code release.
Cite this book
Reference
Citation details
Cite or share this article.
BIBTEXAcademic
@book{brenndoerfer2025mlfromscratch,
author = {Michael Brenndoerfer},
title = {Machine Learning from Scratch},
year = {2025},
url = {https://mbrenndoerfer.com/books/machine-learning-from-scratch},
publisher = {mbrenndoerfer.com},
note = {Accessed: 2026-10-24}
}
APAAcademic
Michael Brenndoerfer (2025). Machine Learning from Scratch. Retrieved from https://mbrenndoerfer.com/books/machine-learning-from-scratch
MLAAcademic
Michael Brenndoerfer. "Machine Learning from Scratch." 2026. Web. October 24, 2026. <https://mbrenndoerfer.com/books/machine-learning-from-scratch>.
CHICAGOAcademic
Michael Brenndoerfer. "Machine Learning from Scratch." Accessed October 24, 2026. https://mbrenndoerfer.com/books/machine-learning-from-scratch.
HARVARDAcademic
Michael Brenndoerfer (2025) 'Machine Learning from Scratch'. Available at: https://mbrenndoerfer.com/books/machine-learning-from-scratch (Accessed: October 24, 2026).
SimpleBasic
Michael Brenndoerfer (2025). Machine Learning from Scratch. https://mbrenndoerfer.com/books/machine-learning-from-scratch