
Foundations, Control, and Representations
Build a first working model, then study the state estimation, control, and representation choices that make its predictions useful for decisions.
Part of World Models Handbook. The online chapters are free to read; the PDF edition is yours to keep, with future updates included.
Build the understanding to reason about world models.
Learn how a system can represent its environment and use predictions to make decisions. The handbook connects the underlying ideas to architectures, planning methods, and evaluation, helping you reason about what a world model can do and where its limits lie.
See what you’ll learnWhat you will learn
- Separate predictive accuracy from decision usefulness.
- Track hidden state with beliefs, filters, and sufficient representations.
- Connect system identification and control to learned world models.
Who this volume is for
- AI and robotics engineers, researchers, graduate students, reinforcement learning practitioners, and technically curious readers who want a coherent path from foundational state estimation and control to modern generative world models.
Prerequisites: Basic Python, probability, linear algebra, and calculus. The examples start with small systems before introducing learned models.
Volume 1 PDF + code
Own this 608-page edition
Keep the PDF and companion notebooks for offline reading, with future updates included. The corresponding chapters stay free online.
- Carefully typeset PDF
- Companion notebooks included
- Future updates and errata included
- Online chapters remain free to read
Volume 1 of 4 · Available now
$29
one-timeVersion 2026.10.0 · 608 pages · secure checkout via Stripe
Delivered by email · Free updates included
Want the complete library?
Access to every volume, delivered as separate PDFs. Companion notebooks included. Save $37 vs. buying available volumes separately.
Get all-volume access · $79On Michael’s writing
Feedback on Michael’s writing
Genuine feedback, with names abbreviated for privacy.
These comments concern Michael’s online writing and other handbooks, not this new PDF edition.
Meet the community“I am enjoying reading your writings. I appreciate the effort you put into it. I want to drop a note of thanks. Thank you!”
“I would like to thank you for your work. Specifically, the chapter: 'LIME Explainability: Complete Guide to Local Interpretable Model-Agnostic Explanations' was extremely helpful for me in understanding LIME and communicating to colleagues. Your work is inspiring and valuable. Thank you for doing it.”
“I just came across your website where your books are available for free to read. Thank you so much for making this available to the public. This will help a lot of people.”
“I found your book Language AI Handbook's explanations of Transformer architectures and production deployment strategies to be exceptionally clear and insightful”
“Your writing feels like a treasure trove of great material. So, thank you for compiling everything on your website!”
“Really appreciate the way you have explained the concepts. Simple to understand, yet builds up knowledge exponentially, all while reinforcing it with examples repeatedly. I've read the last two chapters `74.Self-attention & 75.Q,K,V` and I've to admit, my concepts have never been clearer. You really are doing an amazing job breaking down maths and complex concepts like a story!”
“Thank you Michael. It's top!”
“I read a few of your blogs on NLP and oh WOW, they really are something, now I exactly know how all of the terms Entropy, Cross-entropy and how it is connected to Perplexity, will definitely read more of your blogs.”
“I came across your collection of online handbooks covering quantitative finance, AI, and machine learning. I wanted to reach out and thank you for making such high-quality material openly available. They are incredibly helpful for my CFA Level II review and for brushing up ahead of ML interviews.”
“Thanks so much for establishing such an amazing community. I appreciate the opportunity to be a learner and a contributor to the community.”
“Thanks for putting cool stuff out into the world.”
“I came across one of your books, Language AI Handbook, and man, I'm loving it. I've read many books but have never found such a comprehensive book on large models that covers everything from start to finish with such depth.”
Read all 12 reader reviews
“I would like to thank you for your work. Specifically, the chapter: 'LIME Explainability: Complete Guide to Local Interpretable Model-Agnostic Explanations' was extremely helpful for me in understanding LIME and communicating to colleagues. Your work is inspiring and valuable. Thank you for doing it.”
“Really appreciate the way you have explained the concepts. Simple to understand, yet builds up knowledge exponentially, all while reinforcing it with examples repeatedly. I've read the last two chapters `74.Self-attention & 75.Q,K,V` and I've to admit, my concepts have never been clearer. You really are doing an amazing job breaking down maths and complex concepts like a story!”
“Thanks so much for establishing such an amazing community. I appreciate the opportunity to be a learner and a contributor to the community.”
“I just came across your website where your books are available for free to read. Thank you so much for making this available to the public. This will help a lot of people.”
“I am enjoying reading your writings. I appreciate the effort you put into it. I want to drop a note of thanks. Thank you!”
“Thank you Michael. It's top!”
“Thanks for putting cool stuff out into the world.”
“I found your book Language AI Handbook's explanations of Transformer architectures and production deployment strategies to be exceptionally clear and insightful”
“Your writing feels like a treasure trove of great material. So, thank you for compiling everything on your website!”
“I came across your collection of online handbooks covering quantitative finance, AI, and machine learning. I wanted to reach out and thank you for making such high-quality material openly available. They are incredibly helpful for my CFA Level II review and for brushing up ahead of ML interviews.”
“I read a few of your blogs on NLP and oh WOW, they really are something, now I exactly know how all of the terms Entropy, Cross-entropy and how it is connected to Perplexity, will definitely read more of your blogs.”
“I came across one of your books, Language AI Handbook, and man, I'm loving it. I've read many books but have never found such a comprehensive book on large models that covers everything from start to finish with such depth.”
Exact contents
18 chapters across 3 parts
Every linked chapter is free to read online.
Part I: The World-Model Idea
- 01Building a First World Model
Develops the topic through a minimal latent dynamics model, prediction, reconstruction, and rollout, planning with imagined trajectories, and the complete training and evaluation loop.
- 02What Is a World Model?
Develops the topic through operational and philosophical definitions, models of observations, states, dynamics, and agents, prediction versus simulation, and boundaries with generative models and simulators.
- 03The World-Model Qualification Test
Develops the topic through state sufficiency, counterfactual prediction, action conditioning, and decision usefulness and closed-loop evidence.
- 04Taxonomies of World Models
Develops the topic through explicit and implicit models, pixel, token, latent, object, and symbolic state, deterministic and stochastic dynamics, and task-specific and general-purpose models.
- 05Intellectual and Cognitive History
Develops the topic through internal models in neuroscience and cognitive science, cybernetics and control, AI planning and model-based learning, and modern generative and foundation models.
- 06Mathematical and Dynamical Foundations
Develops the topic through random variables and conditional distributions, state transitions and trajectories, fixed points, stability, and attractors, and information, compression, and sufficient statistics.
Part II: Inference, Decisions, and Control
- 01Markov and Partially Observable Decision Processes
Develops the topic through MDPs, POMDPs, and controlled dynamical systems, states, actions, observations, rewards, and horizons, Markov assumptions and their failure, and policies, returns, and value functions.
- 02Bayesian Filtering and Belief States
Develops the topic through Bayes filters and recursive state estimation, Kalman and extended Kalman filters, particle filters, and learned belief-state updates.
- 03System Identification
Develops the topic through identifiability and excitation, parametric and nonparametric dynamics, subspace and prediction-error methods, and neural system identification from trajectories.
- 04Classical Planning and Optimal Control
Develops the topic through state-space search, dynamic programming and Bellman equations, linear-quadratic control, and trajectory optimization and constraints.
- 05Control as Inference and Active Inference
Develops the topic through probabilistic control objectives, optimality variables and graphical models, free energy and expected free energy, and connections and distinctions between frameworks.
- 06Model-Based Reinforcement Learning
Develops the topic through learning versus using a dynamics model, planning, policy optimization, and value learning, sample efficiency and model bias, and Dyna-style integration of real and imagined data.
Part III: Representing Agents and Worlds
- 01Observations, Sensors, and Multimodal Data
Develops the topic through vision, audio, language, touch, and proprioception, sensor calibration and synchronization, missing, delayed, and noisy observations, and multimodal fusion and alignment.
- 02Sufficient State and State Abstraction
Develops the topic through predictive and control-sufficient state, bisimulation and behavioral equivalence, minimal state and nuisance invariance, and task-aware representation learning.
- 03Latent-Variable Representations
Develops the topic through autoencoders and variational inference, discrete and continuous latents, posterior collapse and information bottlenecks, and latent geometry and semantic organization.
- 04Temporal State and Memory
Develops the topic through recurrent memory and temporal context, short-term versus long-term state, episodic and external memory, and non-Markov environments and state aliasing.
- 05Object-Centric and Relational Worlds
Develops the topic through object discovery and slot representations, relations, graphs, and message passing, identity, permanence, and tracking, and interaction networks and compositional scenes.
- 06Self-Models, Body Schemas, and Affordances
Develops the topic through agent-environment boundaries, body schemas and kinematic state, capabilities, constraints, and affordances, and modeling other agents and theory of mind.
Choose the edition that fits your next step
Volume 1: Foundations, Control, and Representations
608 pages focused on this subject, with future updates included.
$29
Choose this volumeThe complete World Models Handbook library
All-volume access with separate volume PDFs and future updates. No combined PDF.
$79
Get all-volume accessExplore the other volumes

Volume 2 · 637 pages
Physics, Architectures, and Learning
Space, Physics, and Causality · World-Model Architectures · Learning World Models at Scale

Volume 3 · 701 pages
Planning, Research Lineages, and Foundation Models
Planning and Agency · Decision-Centric Research Lineages · Foundation and World-Action Models

Volume 4 · 713 pages
Applications, Evaluation, and Reliable Deployment
Application Domains · Evaluation and Understanding · Reliable World Models
Author and edition details
About the author and this edition

Michael Brenndoerfer
Michael has spent more than a decade working across software engineering, data, AI, and business. He writes to understand difficult ideas more deeply and to share what he learns in a clear, practical way.
- Edition
- First edition
- Published
- Last reviewed
Version history
Kept current, not frozen in time
Library access includes every future volume and update. When a volume changes, its updated PDF appears in My books at no extra cost.
Current release
Edition 2026.10.0
First edition: 72 chapters across four standalone volume PDFs, with executable companion notebooks and perpetual updates.