Intellectual and Cognitive History › Internal Models in Brains and Minds

Long before anyone trained a neural network to predict the next frame, biologists and psychologists were cataloguing evidence that brains run predictive models of the world and of the body's own actions. This evidence accumulated over more than a century, across perception, motor control, spatial navigation, memory, and reasoning, and it converged on a conclusion that was far from obvious: the brain is not merely a reactive device that maps stimuli to responses, but also a predictive device that generates expectations and gives prediction errors a central role. This section covers the major strands of that evidence, because each strand maps onto a design decision you will face when building artificial world models. The mapping is worth making explicit up front. Unconscious inference foreshadows the encoder-decoder structure of latent-variable models. Efference copies motivate forward models used for state estimation and action-conditioned prediction. Cognitive maps anticipate state abstractions and successor representations. Mental simulation resembles imagined rollouts and planning in latent space. Predictive processing motivates hierarchical generative models trained by minimizing prediction error. The subsections trace these correspondences; later parts of the book develop the engineering versions in detail.

Hermann von Helmholtz developed the idea of unconscious inference across his Handbook of Physiological Optics, published in parts from 1856 to 1867. The retinal image is ambiguous: the same pattern of light can be compatible with many external scenes, and retinal motion can come from an object or from the observer's eye. Helmholtz argued that perception resolves such ambiguity by combining sensory evidence with learned expectations about its likely causes. "Unconscious" mattered: the inference was not deliberate reasoning, but a rapid process below awareness. The proposal was not Bayesian in modern notation, yet it supplied the historical structure that later Bayesian accounts of perception made explicit.

In modern notation, the brain estimates a latent cause $z$ from a sensation $x$ by weighting evidence against priors, exactly as Bayes' rule prescribes:

Read this equation slowly, because each factor corresponds to a recurring component of probabilistic world models. The likelihood $p(x \mid z)$ is the observation component: it describes how unobservable causes produce observable sensations. Together, the prior $p(z)$ and likelihood $p(x \mid z)$ define the joint generative model $p(x,z)=p(x \mid z)p(z)$. The posterior $p(z \mid x)$ is the perceiver's updated belief after seeing the data; the omitted normalizer $p(x)$ is the evidence that makes the posterior integrate to one. Perception, in this view, is not the beginning of cognition but the middle of it: it stands between a body of prior knowledge and the actions that knowledge supports.

Helmholtz wrote before Bayes' rule became a tool of psychology, and he used the word "inference" cautiously, but the structural claim is unmistakable: perception is a prediction problem run in reverse. The perceiver holds a model of how causes produce sensations, and perception is inference about causes given sensations. It took roughly 115 to 140 years for this idea to become computational through Bayesian models of perception in the 1980s and 1990s, depending on which of Helmholtz's writings marks the start and which early computational work counts as the arrival. The conceptual origin, however, lies in Helmholtz's nineteenth-century account.

This framing matters for world models because it separates two things that beginners often conflate: the generative direction (causes produce observations) and the inferential direction (observations are used to infer causes). Probabilistic latent-state architectures usually need both. The forward direction lets you simulate: given a hypothesized state of the world, roll out what you would see. The inverse direction lets you update your beliefs when reality disagrees with your predictions: given what you saw, revise your estimate of the state. An agent equipped with only the forward direction can imagine but cannot update its latent state from observations; an agent equipped with only the inverse direction can encode observations but cannot roll consequences forward. Kalman filters, variational autoencoders, and Dreamer's recurrent state-space model arrange these directions differently. Part II develops Bayesian filtering and belief states, while Part III develops latent-variable representations.

Move your eyes right now, from one word to the next. The visual scene sweeps across your retina, yet you do not experience the world as lurching. Now poke your eye gently through the eyelid: broadly similar retinal displacement is experienced as the world jumping. The inputs are not literally identical, because self-generated saccades also engage suppression and other preparatory mechanisms. The decisive distinction is that the first motion was commanded and predictable, while the second was imposed from outside. Information about the motor command helps the nervous system interpret the resulting sensory change.

In the 1950s, Erich von Holst and Horst Mittelstaedt formulated the reafference principle, and Roger Sperry independently described the same mechanism in fish, to explain this. When the brain issues a motor command, it also sends a copy of that command, an efference copy, to sensory areas. This copy lets the brain predict the sensory consequences of its own action and cancel out the predictable part, the reafference, leaving only the surprising part to be perceived. The elegance of the arrangement is that it solves two problems at once. It prevents the organism from being flooded by self-generated sensation, which would drown out signals from the external world. And it converts every self-produced movement into a test of the forward model: if the predicted sensory consequence matches the actual one, nothing needs to be reported; if it does not match, the mismatch is precisely the information the organism needs.

This is a forward model in the strictest sense: a mapping from current state and action to predicted next sensation. Work by Daniel Wolpert, Mitsuo Kawato, and colleagues in the 1990s made forward and inverse models a central computational account of human motor control; Wolpert and Kawato's 1998 paper is a primary statement of that framework. Before an object leaves a surface, people scale lifting forces to its expected weight; Johansson and Westling's unexpected-weight experiments showed the resulting error when that expectation was wrong. A deceptively light suitcase can therefore make the arm rise faster than intended. The cerebellum has been implicated in these predictions and in adapting movements when action and sensation no longer match, although that evidence does not make every cerebellar computation a forward model.

A charming demonstration of the same machinery: self-produced touch is usually much less ticklish than externally produced touch. When you try to tickle your own palm, your forward model predicts the sensation, and the brain attenuates it. Blakemore, Wolpert, and Frith's 1998 paper explicitly reports using fMRI and found reduced responses to self-produced touch; the bibliographic record gives Nature Neuroscience 1:635-640 and DOI 10.1038/2870. In a 1999 behavioral experiment, delaying the touch or perturbing its trajectory made it feel more ticklish as predicted and actual feedback diverged. The demonstration matters beyond its charm: the brain's response depends not just on the sensation but on how well it was predicted. Prediction quality modulates perception itself, a principle that returns in modern form as precision-weighting in predictive processing and uncertainty-aware filtering in engineering.

Notice what this buys the organism. A forward model converts a slow feedback loop (act, wait for sensation, correct) into a fast internal one (act, predict sensation, correct before the sensation even arrives). The external loop is limited by nerve conduction velocity, sensory transduction, and muscle activation, all of which are physically slow. The internal loop runs at the speed of neural computation, which is much faster. This is precisely the argument for model-based control that we will formalize in the cybernetics section, and it is the same argument behind model-predictive control and imagined rollouts in Part VII. When an engineer builds a Smith predictor or an MPC controller, she is installing an artificial efference copy: a prediction of what the plant will do in response to the command, available before the plant's sluggish dynamics deliver the real answer.

In the 1930s and 1940s, Edward Tolman ran experiments that behaviorism could not explain. The dominant psychology of the day held that learning was the formation of stimulus-response associations, stamped in by reward. On that view, an animal learns nothing without reinforcement, and what it learns is a habit, a fixed response to a fixed cue. Tolman's rats misbehaved in both respects. Rats allowed to wander a maze with no reward, and later rewarded at a goal location, navigated almost as well as rats trained with reward from the start. This latent learning meant the unrewarded rats had learned something about the maze's structure, not just a stimulus-response habit. They had been building knowledge the whole time, silently, waiting for a reason to use it. Other experiments showed detour behavior: rats familiar with a maze would take novel shortcuts when a barrier blocked the trained route, behavior that no chain of practiced stimulus-response pairs could produce. Tolman concluded in his 1948 paper "Cognitive maps in rats and men" that animals build internal representations of spatial layout, and that behavior is planned against these representations rather than emitted as fixed responses.

Tolman's proposal remained physiological speculation for more than two decades after his 1948 paper. Psychology had no tools for finding a map in tissue, and behaviorism remained hostile to internal states of any kind. Then in 1971, John O'Keefe and Jonathan Dostrovsky discovered place cells: neurons in the rat hippocampus with location-selective firing fields. Place-cell activity is not reducible to a single visual landmark or a particular sequence of turns; it relates to location, an abstract variable that multiple sensory streams and action sequences can imply, even as fields adjust with context and approach direction and remap entirely under sufficiently large contextual changes, as Muller and Kubie (1987) and Bostock, Muller, and Kubie (1991) demonstrated under controlled changes and experience. In 1978, O'Keefe and Lynn Nadel published The Hippocampus as a Cognitive Map, arguing that this circuit implements Tolman's construct. In 2005, more than three decades after the place-cell discovery, Edvard Moser, May-Britt Moser, and colleagues reported grid cells in the entorhinal cortex: neurons with strikingly regular hexagonal firing fields that appear to provide a metric coordinate system for space. O'Keefe and the Mosers shared the 2014 Nobel Prize in Physiology or Medicine for this work.

Two aspects of this research program deserve your attention as a world-model builder:

• The map encodes structure, not observations. A place cell can retain a spatial firing field across changes in illumination, approach direction, and some rearrangements, although sufficiently large contextual changes can cause remapping. The hippocampus therefore abstracts away from much raw sensory input toward state variables that support prediction and planning. This is an early biological analogue of what Chapter 2 calls a decision-sufficient representation: preserve variables that matter for decisions and discard others. A place field is not a photograph of the room. The later MuZero system made a related engineering choice by learning planning-relevant latent states without reconstructing every observation detail; it reappears in this chapter's history of learned models.
• The map supports inference beyond practiced routes. Tolman's detour and shortcut experiments showed that route memorization alone could not explain flexible navigation. Morris and colleagues' 1982 water-maze lesion experiments became an influential demonstration that hippocampal damage impairs flexible spatial navigation, although the strength of the deficit depends on the task and does not justify a blanket claim about every novel shortcut. Replay supplies a second clue. Wilson and McNaughton reported reactivation during sleep in 1994; Foster and Wilson recorded awake reverse replay in 2006; later work, including Gupta and colleagues in 2010, found sequences that could represent paths the animal had not just traversed. These results have motivated comparison with Dyna, the model-based reinforcement learning architecture introduced below: real experience updates a model, and internally generated sequences can support later decisions. The analogy is useful, but the recordings do not by themselves prove that replay performs a policy update.

The cognitive-map story also has a modern computational afterlife. Peter Dayan introduced the successor representation in 1993 in the context of temporal-difference learning. It is a matrix indexed by a current state and a possible successor state, with each entry recording the discounted expected occupancy of that successor under the agent's current policy: change the policy and the matrix changes with it. Kim Stachenfeld and colleagues showed in 2017 that successor representations predict features of hippocampal place-field geometry, including fields that skew opposite the direction of travel and distort around barriers as transition structure changes. In 2018, Timothy Behrens, Timothy Muller, James C. R. Whittington, and colleagues argued that cognitive maps may generalize beyond physical space to abstract relational structure. The engineering lesson is that a useful model can represent predictive relationships among states rather than reconstruct raw observations.

Cognitive science in the 1980s developed the idea that reasoning itself is simulation. Philip Johnson-Laird's Mental Models (1983) argued that people reason by constructing small internal models of a situation and inspecting them, rather than by applying logical rules to propositions. Asked whether "all the flowers are roses" follows from "some of the flowers are roses," people do not consult a proof system; they picture a garden with some roses and check whether other flowers could coexist with the statement. The mental model tradition explained systematic patterns in human reasoning errors as artifacts of what people's internal simulations leave out, which is a very different account from logic-based theories of thought. In cognitive psychology, the "theory theory" tradition associated with Alison Gopnik and Henry Wellman held that children learn causal models of the world and revise them like little scientists testing hypotheses. On this view, developmental learning is not accumulation of associations but theory change: the child's causal model of objects, minds, and living things undergoes revisions that resemble paradigm shifts, complete with periods of confusion when the old model fails and the new one is not yet stable.

In 2002, cognitive neuroscientist Georg Hesslow proposed that conscious thought can be understood as simulation of behavior and perception: when you deliberate, you covertly recruit some of the motor and perceptual machinery that would be engaged if you acted. Deliberation, on this account, is not symbol manipulation wholly detached from the sensorimotor system; it can be the sensorimotor system running offline, with overt action inhibited. Memory research supplied a complementary line of evidence. Schacter and Addis argued in 2007 that episodic memory is constructive and can recombine fragments of past experience into imagined futures. In the same year, Hassabis and colleagues reported that several patients with hippocampal amnesia had difficulty constructing coherent novel experiences, although later work showed that this deficit is not universal across all amnesic patients. Memory, in this view, is less like a recording archive than a parts bin for simulation. Endel Tulving had earlier called the capacity to mentally travel in subjective time "chronesthesia."

The simulation hypothesis gives a cognitive-science answer to the question "why do agents need world models?" Because deliberation can take the form of model-based rollouts. When you decide whether to say something in a meeting, you may simulate the conversation; when a chess player looks ahead, she runs a model of the game; when you route around traffic, you simulate the drive. The next two traditions pursued intelligent action through different mechanisms: cybernetics emphasized feedback, while AI planning emphasized search over predicted consequences. Together they show that explicit simulation is one tool among several, not the definition of intelligent action. It is worth being precise about what the cognitive evidence does and does not establish. It establishes that humans use simulation and that simulation can recruit machinery also used in perception and action. It does not establish that simulation is always beneficial: human simulation is also the source of anxiety, rumination, and systematic forecasting errors, and people often decide better when they rely on fast habitual responses than when they deliberate badly. The engineering translation is not "always simulate" but "simulate when the model is good enough and the stakes justify the compute," a conditional that recurs throughout this book.

The final cognitive strand assembles Helmholtz's inference, motor prediction, and attention into a single architectural proposal about the cortex. David Mumford suggested in the early 1990s that the cortex's hierarchical structure supports generative feedback: higher areas send predictions down, lower areas send prediction errors up. A motivating anatomical asymmetry was that feedback connections between cortical areas are abundant and, in many measured pathways, outnumber feedforward connections. Exact ratios vary by area, layer, and measurement method, but the prevalence of backward pathways is hard to reconcile with a purely feedforward account of perception. Rajesh Rao and Dana Ballard gave this idea a concrete computational form in 1999, showing that a hierarchical predictive-coding network reproduced response properties of visual neurons, including extra-classical receptive-field effects. Karl Friston, James Kilner, and Lee Harrison's 2006 account developed the idea into the free-energy principle, attempting to unify perception, action, learning, and attention as facets of prediction-error minimization, and Andy Clark's 2013 paper "Whatever next?" and 2016 book Surfing Uncertainty brought it to a wide audience.

Three claims in this framework matter for our purposes. First, action and perception share a loop: action is the control solution to the prediction problem, an idea that anticipates the control-as-inference view we will develop in Part II. Where classical accounts treat perception and action as separate pipelines, predictive processing treats them as two ways of resolving the same discrepancy between prediction and sensation: change the belief, or change the world. Second, one influential account holds that attention tracks uncertainty: precision-weighting amplifies prediction errors from reliable channels and discounts errors from noisy ones. This gives attention a possible computational role and anticipates the uncertainty-weighted fusion at the heart of the Kalman filter. Third, and most cautiously, the framework's grander ambitions, such as explaining all of cognition as free-energy minimization, remain contested. Critics have argued that the principle is too flexible to falsify, that prediction-error minimization cannot be the whole story of cortical function, and that the mathematical formulations admit multiple incompatible readings. The durable architectural idea is a hierarchy of generative predictions and corrective error signals. Residual connections, variational inference, and encoder-decoder loops are not direct implementations of predictive coding, but they can be read as engineering echoes of this predict-and-correct pattern.