Forecasting Baselines, Temporal Validation, and Backtesting
Build naive, seasonal-naive, and drift baselines, then validate forecasts with rolling-origin backtesting, horizon-specific errors, and honest reporting.
702 items · Page 1 of 15
Create a free account to keep your reading organized, join thoughtful discussions, and get more from every chapter.
Free to join · Takes less than a minute
Why
I created this space so readers can learn together, ask questions, and make sense of difficult ideas.

Build naive, seasonal-naive, and drift baselines, then validate forecasts with rolling-origin backtesting, horizon-specific errors, and honest reporting.
Genetic algorithms from scratch: encoding, tournament selection, order crossover, mutation, elitism, diversity, and budget-matched routing comparisons.
Build local search and simulated annealing from scratch on an eight-customer routing instance, then compare them across 20 seeds against the enumerated optimum.
Build distribution-free prediction intervals and label sets with split conformal prediction, including finite-sample quantiles and marginal coverage guarantees.
Probability calibration for classifiers: reliability diagrams, Brier score, log loss, sigmoid and isotonic recalibration, and practical monitoring.
Build random survival forests from scratch: log-rank splits, Nelson-Aalen leaves, censoring-aware risk sets, OOB scoring, and comparison with Cox models.
A production example links objectives, domains, and constraints to feasible plans, LP and integer optima, optimality gaps, and independent audits.
Build Differential Evolution from scratch: differential mutation, binomial crossover, greedy selection, bounds, and budget-matched benchmarks on Rastrigin.
Build a one-hidden-layer multilayer perceptron from scratch: derive forward and backward passes, check gradients, and train with mini-batches.
Gaussian mixture models cluster with soft assignments. EM derivations and NumPy code explain covariance geometry, stable fitting, and component selection.
Score censored survival models with concordance and IPCW Brier scores. Learn to compare Cox, Weibull AFT, and Kaplan-Meier forecasts honestly.
Build LDA and QDA classifiers from Gaussian class models and Bayes' rule, with covariance estimation, shrinkage, and NumPy implementations.
Learn how naive Bayes classifiers work: Bayes decision rule, Gaussian and multinomial event models, smoothing, log-sum-exp, and NumPy implementation.
Build kNN from scratch, covering distance metrics, majority voting, local averaging, feature scaling, distance weighting, and the curse of dimensionality.
Learn how the Cox proportional hazards model estimates covariate effects on hazards using partial likelihood, with NumPy implementation and diagnostics.
Learn how support vector machines and support vector regression work, from margins and hinge loss to kernels, dual coefficients, epsilon tubes, and scikit-learn implementation.
Learn Kaplan-Meier and Nelson-Aalen estimation for right-censored data, covering risk sets, ties, Greenwood variance, medians, and NumPy implementation.
Learn quantile regression from first principles: pinball loss, linear programming, conditional quantiles, crossing diagnostics, and Python implementation.
Learn how AFT models fit censored event-time data, interpret coefficients as time ratios, and compare Weibull and log-normal survival curves.
Learn how survival analysis handles right-censored durations, defines hazard and survival functions, and builds the censored-data likelihood from scratch.
Master hyperparameter search and nested cross-validation. Compare grid and random search, diagnose selection bias, and evaluate tuning procedures honestly.
Learn task-matched validation with grouped, stratified, and temporal splits, out-of-fold predictions, and preprocessing within the training boundary.
Learn how gradient descent updates parameters on quadratic objectives, how curvature and learning rates control convergence, and how to verify least-squares fits in NumPy.
Learn maximum likelihood estimation from scratch, including Bernoulli and Gaussian estimators, log-likelihood, score, curvature, and NumPy/SciPy implementation.
Learn generalized additive models from scratch: spline bases, difference penalties, link functions, identifiability, cross-validated smoothing, and NumPy fits.
Learn how confusion matrices, precision, recall, ROC AUC, and average precision guide threshold selection under class imbalance and decision costs in practice.
Frontier questions and research roadmaps for world models: generality contracts, causal structure, persistent memory, and decision-relevant evaluation.
A reproducible sim-to-real workflow covering structural mismatch, domain randomization, delay ablations, residual monitoring, and gated continual correction.
Distributed training, checkpointing, and stateful rollout serving for world models, with latency, memory, and energy budgets for deployment.
World model security, privacy, and governance: poisoning, extraction, membership leakage, dual-use simulation, documentation, and access control.
Stress testing, calibrated uncertainty, risk-sensitive planning, and runtime assurance for safe control with imperfect world models.
World models fail through compounding error, objective misspecification, planner exploitation, state aliasing, forgetting, and hallucinated dynamics.
Probes, TCAV, rollouts, and component substitution show what a world model represents, what its planner uses, and where causal claims remain unsupported.
Benchmark selection, leakage checks, ablations, and compute reporting connect world-model claims to reproducible experimental evidence.
Evaluate composed model-planner-policy systems through closed-loop return, sample efficiency, planning horizon tradeoffs, exploitation, and transfer protocols.
Test world-model state, physics, causality, and memory with probes, occlusion, energy checks, and paired interventions; learn what each diagnostic can prove.
Score predicted observations with MSE, PSNR, SSIM, LPIPS, FID, and FVD. Protocols, conditioning, best-of-K, and human judgment shape the results.
World-model evaluation uses a five-rung ladder covering one-step prediction, open-loop rollout, intervention, closed-loop planning, and transfer.
Scientific, medical, and social world models connect prediction, simulation, and counterfactual inference under uncertainty and domain constraints.
Model web, desktop, and code-execution state as partial observations, then verify actions, correct beliefs, and separate prediction error from task success.
Examine driving world models through bird's-eye-view and occupancy state, other-agent prediction, closed-loop simulation, and rare-event safety planning.
Model legged locomotion, whole-body control, and map-based navigation with hybrid contact dynamics, terrain uncertainty, and sim-to-real transfer.
Build a miniature planar pushing world model in Python: contact features, Ridge regressors, rollout error, random-shooting plans, and affordance maps.
Compare Atari, board games, and open worlds as learned game engines. Measure one-step prediction, recursive rollout drift, and model-predictive control.
VLA and world-action models fuse vision, language, and control, tokenize robot actions, and transfer across embodiments under closed-loop evaluation.
Examines NVIDIA's Cosmos world foundation models: video tokenization, physical AI data curation, conditional generation, guidance calibration, and simulation.
Genie discovers latent actions from unlabeled video and builds interactive generated environments, with tests for control, rollout, and persistence.
Can video generators serve as world models? Tests cover action-conditioned prediction, a qualification ladder, occlusion memory, and closed-loop planning.
Create a free account to keep your reading organized, join thoughtful discussions, and get more from every chapter.
Free to join · Takes less than a minute
Why
I created this space so readers can learn together, ask questions, and make sense of difficult ideas.
