Machine Learning from Scratch

Complete guide to data classification - quantitative, qualitative, discrete & continuous

Complete guide to summarizing and understanding data with measures of central tendency, variability, and distribution shape

Foundation of statistical reasoning covering random variables, probability distributions, expected value, variance, and conditional probability

Foundation of statistical inference covering convergence behavior, sample size requirements, and practical applications in data science

Complete guide to sampling theory and methods covering simple random sampling, stratified sampling, cluster sampling, sampling error, and uncertainty quantification

Complete guide to covariance, correlation, and regression analysis covering how to measure, model, and interpret variable associations

Complete guide to normal, t-distribution, binomial, Poisson, exponential, and log-normal distributions with practical applications

Complete guide to histograms, box plots, and scatter plots for exploratory data analysis

Complete guide to data quality and outliers covering measurement error, bias, missing data, and imputation

Complete guide to drawing conclusions from data covering point and interval estimation, confidence intervals, hypothesis testing, and p-values

Complete guide to building and evaluating predictive models covering model fit metrics, bias-variance tradeoff, and cross-validation

The fundamental metric for measuring regression model performance and prediction accuracy

Understanding the coefficient of determination and model fit metrics

Normalizing features for fair comparison in machine learning algorithms

Min-max scaling to transform features to a common [0, 1] range for neural networks and distance-based algorithms

Foundation of linear regression and OLS estimation covering linearity, independence, homoscedasticity, normality, and practical testing methods

Understanding the impact of multicollinearity on regression models

Mathematical foundations, formulas, and step-by-step implementation

Vector notation and matrix operations for regression

Extending linear regression to multiple predictors

L1 penalty for feature selection and overfitting prevention

L2 penalty for handling multicollinearity and overfitting

Combining L1 and L2 penalties for optimal regularization

Modeling non-linear relationships with polynomial features

Extending linear regression to non-normal distributions

Binary classification using the logistic function

Flexible non-parametric regression using piecewise polynomials

Modeling count data with Poisson distribution

Multi-class classification extension of logistic regression

Decision trees with greedy splitting algorithms

Ensemble method combining multiple decision trees with bagging

Gradient boosting for improved predictive performance

Optimized gradient boosting with advanced regularization techniques

Fast gradient boosting with leaf-wise tree growth

Gradient boosting with categorical feature handling

Unsupervised anomaly detection using random trees

Unified framework for model interpretability

Local model explanations for individual predictions

Dimensionality reduction and feature extraction

Non-linear dimensionality reduction preserving local and global structure

Non-linear visualization technique

Partitioning data into k clusters using centroid-based approach

Density-based clustering for arbitrary shaped clusters

Hierarchical density-based clustering with varying density

Tree-based clustering with agglomerative or divisive methods

Classical time series forecasting with trend and seasonality

Autoregressive integrated moving average with seasonal components

Facebook's forecasting tool for business time series with holidays

Neural basis expansion analysis for interpretable time series forecasting

Neural hierarchical interpolation for time series forecasting

Constraint programming for employee scheduling and rostering

Mixed integer linear programming for production planning

Network flow optimization for resource allocation

Vehicle routing problem with time windows for logistics

Quadratic programming for portfolio optimization and risk management