Machine Learning from Scratch

Complete guide to data classification - quantitative, qualitative, discrete & continuous

Complete guide to summarizing and understanding data with measures of central tendency, variability, and distribution shape

Foundation of statistical reasoning covering random variables, probability distributions, expected value, variance, and conditional probability

Foundation of statistical inference covering convergence behavior, sample size requirements, and practical applications in data science

Complete guide to sampling theory and methods covering simple random sampling, stratified sampling, cluster sampling, sampling error, and uncertainty quantification

Complete guide to covariance, correlation, and regression analysis covering how to measure, model, and interpret variable associations

Complete guide to normal, t-distribution, binomial, Poisson, exponential, and log-normal distributions with practical applications

Complete guide to histograms, box plots, and scatter plots for exploratory data analysis

Complete guide to data quality and outliers covering measurement error, bias, missing data, and imputation

Complete guide to drawing conclusions from data covering point and interval estimation, confidence intervals, hypothesis testing, and p-values

Complete guide to building and evaluating predictive models covering model fit metrics, bias-variance tradeoff, and cross-validation

Foundation of hypothesis testing covering p-values, null and alternative hypotheses, one-sided vs two-sided tests, and test statistics

Mathematical equivalence between confidence intervals and hypothesis tests, test assumptions, and choosing between z and t tests

Complete guide to z-tests including one-sample, two-sample, and proportion tests

Complete guide to t-tests including one-sample, two-sample (pooled and Welch), paired tests, assumptions, and decision framework

F-distribution, F-test for comparing variances, F-test in regression, and nested model comparison

One-way ANOVA, post-hoc tests, assumptions, and when to use ANOVA

Understanding false positives, false negatives, statistical power, and the tradeoff between error types

Power analysis, sample size determination, MDE calculation, and avoiding underpowered studies

Cohen's d, practical significance, interpreting effect sizes, and why tiny p-values can mean tiny effects

Family-wise error rate, false discovery rate, Bonferroni correction, Holm's method, and Benjamini-Hochberg procedure

Practical reporting guidelines, summary of key concepts, test selection parameters table, multiple comparison corrections table, and scipy.stats functions reference

The fundamental metric for measuring regression model performance and prediction accuracy

Understanding the coefficient of determination and model fit metrics

Normalizing features for fair comparison in machine learning algorithms

Min-max scaling to transform features to a common [0, 1] range for neural networks and distance-based algorithms

Foundation of linear regression and OLS estimation covering linearity, independence, homoscedasticity, normality, and practical testing methods

Understanding the impact of multicollinearity on regression models

Mathematical foundations, formulas, and step-by-step implementation

Vector notation and matrix operations for regression

Extending linear regression to multiple predictors

L1 penalty for feature selection and overfitting prevention

L2 penalty for handling multicollinearity and overfitting

Combining L1 and L2 penalties for optimal regularization

Modeling non-linear relationships with polynomial features

Extending linear regression to non-normal distributions

Binary classification using the logistic function

Flexible non-parametric regression using piecewise polynomials

Modeling count data with Poisson distribution

Multi-class classification extension of logistic regression

Decision trees with greedy splitting algorithms

Ensemble method combining multiple decision trees with bagging

Gradient boosting for improved predictive performance

Optimized gradient boosting with advanced regularization techniques

Fast gradient boosting with leaf-wise tree growth

Gradient boosting with categorical feature handling

Unified framework for model interpretability

Local model explanations for individual predictions

Dimensionality reduction and feature extraction

Non-linear dimensionality reduction preserving local and global structure

Non-linear visualization technique

Partitioning data into k clusters using centroid-based approach

Density-based clustering for arbitrary shaped clusters and noise detection

Hierarchical density-based clustering with varying density

Tree-based clustering with agglomerative or divisive methods

Unsupervised anomaly detection using random trees

Classical time series forecasting with trend and seasonality

Autoregressive integrated moving average with seasonal components

Facebook's forecasting tool for business time series with holidays

Neural basis expansion analysis for interpretable time series forecasting

Neural hierarchical interpolation for time series forecasting

Constraint programming for employee scheduling and rostering

Mixed integer linear programming for production planning

Network flow optimization for resource allocation

Vehicle routing problem with time windows for logistics

Quadratic programming for portfolio optimization and risk management