Causal Models and Interventions

A building-management agent watches months of logs from a climate system. On average in the logs, higher cooling is associated with a temperature drop smaller than the engineers' physics model predicts. Operators turn cooling up mostly on hot days, so strong cooling is systematically paired with strong heat loads. If the agent interprets that association as the response to an intervention, it learns the wrong slope and can choose the wrong plan.

Notice how subtle this failure is. The log is enormous by the standards of this example: months of data and thousands of episodes. A regression can still be the best linear predictor available within the distribution the operators generated while explaining only part of the outcome variance. Ordinary held-out error from the same logging policy does not test the quantity the planner needs: the response to a deliberately changed action. The failure appears when the model is used, especially at action values and in situations that the operators' behavior policy rarely paired together. This separates predictive fit under one policy from decision-making validity under another; a predictive evaluation can reveal weak fit, but it cannot by itself certify an interventional effect.

This is the central problem of the chapter. Models trained only for observational prediction learn relations under the data-collection process they experienced. Planning, by contrast, evaluates deliberately selected actions, which can be represented as interventions on an action node. A predictor may generalize across a policy change when its learned relation and support remain valid, or fail when they do not. Correctly specified and identified causal mechanisms provide semantics for such changed-policy queries; writing down a causal graph alone does not remove misspecification, nonidentifiability, or limited support.

The word "mechanism" is doing heavy lifting here, so it is worth pausing on what it buys us. A joint distribution determines probabilities of observational events (or densities over regions for continuous variables), but it says nothing by itself about which outcomes would arise if you reached in and changed something. A set of mechanisms, by contrast, is compositional. Each rule has its own identity, inputs, and domain of validity, and knowing the rules separately can support reasoning about swapping one out while leaving the others in place. This is useful for agents because each action replaces the default rule for what action happens next with the agent's choice. Those intervention semantics are additional structure; they are not recoverable from an observational joint distribution alone.

In this chapter we build the causal vocabulary step by step: structural causal models, the distinction between seeing and doing, counterfactual reasoning, confounding and adjustment, and the properties of invariance and transportability that make causal knowledge reusable. We close with causal representation learning, the open research problem of getting causal structure out of raw sensors, which connects directly to the latent world models of Part III.

Pearl's hierarchy of causal queries gives a compact way to see what is at stake. Causal queries in this framework fall into three rungs. Moving upward generally requires additional assumptions about how the world generates data; more passive data alone cannot establish assumptions that are observationally indistinguishable.

The distinction matters because different parts of an agent live on different rungs. An observational video predictor answering "what does the scene look like next?" is a Rung 1 machine. A model-predictive controller asks "what happens to the trajectory if I apply action sequence $a_{t:t+H}$?", a Rung 2 question when its action-conditioned model has valid intervention semantics. Diagnosis such as "would the collision have happened had the planner chosen the other lane?" is a Rung 3 question. Supporting all three requires more structure than association alone and generally more than intervention alone; a counterfactual-capable SCM normally also supports the lower-rung queries.

Here is the uncomfortable fact that motivates the chapter: planning poses Rung 2 queries, while passive prediction is a Rung 1 task. Model-based reinforcement learning can exploit model error, but confounding is only one source alongside approximation error, finite data, recursive rollout, and limited support. Optimization may seek favorable model errors; whether it does so depends on the planner and error landscape. Causal structure can help distinguish stable mechanisms from collection artifacts when that structure is justified and identified.

Many baseline MBRL systems minimize average prediction loss and use ensembles or penalties to discourage uncertain rollouts. Error is not uniform: finite-sample variation, approximation error, support gaps, and confounding behave differently under optimization. Noise need not average away, and a planner need not exploit every bias. Randomized tests, sensitivity analysis, robust bounds, and structural knowledge can all help separate or constrain these errors; a complete causal decomposition is not always available.

The basic object is deliberately simple: a collection of variables, each assigned its own generating rule. The simplicity is the point. Rather than trying to encode causal knowledge in constraints on a joint distribution, which tends to be both unnatural and fragile, we encode it in the way the variables are constructed, one at a time, from their causes.

A few reading notes on this definition. The endogenous/exogenous split is a modeling choice, not a property of nature: exogenous variables are simply the factors we have chosen not to model, and the quality of an SCM depends on putting the right things on each side of that line. The exogenous variables make each mechanism deterministic despite a stochastic world: all randomness is pushed into $\mathcal{U}$, so that conditional on $u$, every endogenous variable is a fixed function of its parents. In a recursive or otherwise uniquely solvable SCM, this determinism gives a fixed exogenous profile exactly one solution under each contemplated intervention, which is what later makes counterfactual questions well defined. The parent sets $\mathrm{pa}(v_i)$ define the causal graph, a directed graph over $V$ with an edge from each parent to each child, and the structural equations together with the graph are the model's causal content.

Two properties do most of the work. First, each mechanism is a separate function with its own exogenous noise: the rule that generates the room temperature is a different object from the rule that generates the operator's cooling decision. Second, for the recursive, acyclic SCMs used in this chapter, the equations have a unique solution. If we also assume the system is Markovian, meaning the exogenous variables are mutually independent, the resulting joint distribution factorizes along the causal graph:

where:

• $v_i$: the realized value of endogenous variable $V_i$; lowercase letters denote observed values, uppercase letters the variables themselves
• $\mathrm{pa}(v_i)$: the values of the parents of $V_i$ in the causal graph (the variables that feed directly into its mechanism $f_i$)
• the product runs over all $n$ endogenous variables, one factor per mechanism

The Markovian assumption is worth dwelling on because it does essential work. Independence of the exogenous variables rules out residual dependence between the disturbances assigned to different mechanisms. Correlated disturbances can indicate an omitted common cause or other misspecification, but independence is a modeling assumption rather than an exact empirical test of completeness.

A Bayesian network encodes this observational factorization, while an SCM adds intervention semantics to its conditional kernels. Under the stated autonomy assumption, a hard intervention replaces the target mechanism and leaves non-target mechanisms fixed. An arbitrary likelihood factorization does not receive that interpretation automatically, although a deep generative model can be designed with causal structure. The distinction is semantic, not merely whether the software contains separate factors.

In the MDPs of Part II, the transition kernel $P(s' \mid s,a)$ can be interpreted as a mechanism when state and action supply the needed conditioning history. Its causal validity also depends on support, consistency, selection, specification, dynamics, and the intervention target. If a recorded state omits a common cause of action and future outcome, the fitted kernel can mix over that cause and fail to identify an action effect. This is one overlap between state insufficiency and confounding, not an equivalence between them.

This connection runs deeper than an analogy. In an MDP, the transition mechanism is queried with an action selected by the agent; the action is externally set relative to the environment's transition equation. In a fixed dataset, however, logged actions were generated by a behavior policy conditioned on its information. Treating them as freely randomized inputs is justified only when the modeled state blocks the relevant common causes and the required action support is present. Complete observed state can remove this confounding under the Markov assumption, but it does not by itself guarantee positivity, correct specification, or immunity to selection bias. Partial or stale state can leave common causes unobserved and bias a fitted transition response.

Let us make this concrete with a small four-variable SCM, the classic sprinkler story. Season $S$ causes rain $R$ and also whether the sprinkler was scheduled to run $T$; rain and sprinkler each cause the grass to be wet $W$.

The mechanisms are:

• $S \sim \text{Bernoulli}(0.5)$, exogenous.
• $R = f_R(S, u_R)$, for example $P(R=1 \mid S=1) = 0.3$ in the dry season and $P(R=1 \mid S=0) = 0.6$ in the wet season.
• $T = f_T(S, u_T)$, say $P(T=1 \mid S=1) = 0.6$ in the dry season and $P(T=1 \mid S=0) = 0.1$ in the wet season.
• $W = f_W(R, T) = R \lor T$ in this deterministic toy model.

Notice the shape of this little graph. Season is a common cause of both rain and sprinkler use, so it sits at the top with two outgoing edges. Rain and sprinkler then converge on wetness, forming a collider at $W$. In this four-variable model, conditioning on season blocks the fork-induced association between rain and sprinkler use, while conditioning on their common effect $W$ can induce an association through the collider.

Each line is a module. If the city installs a smart controller that changes the sprinkler policy $f_T$, the model assumes that the rain and wetting mechanisms are untouched. This autonomy is a central causal-modeling assumption, not a consequence of the data.

It is worth being clear about the epistemic status of this assumption. It cannot be verified from a single dataset, because a single dataset is one draw from one configuration of mechanisms, and many different mechanism assignments produce identical data. What can support it is domain knowledge and evidence from changes across environments. In the toy sprinkler model, for example, we explicitly assume that replacing the controller does not alter the rain mechanism; a real sensor-driven controller could violate that separation. When modularity fails, a change in one place can alter statistics that the model expected to remain stable. Invariance tests can reveal some such violations when the available environments supply informative changes, but they cannot guarantee that every spurious relation will be exposed.

An intervention is the surgical act of reaching into a system and replacing one mechanism with another. Every non-target mechanism and the distribution of the exogenous variables are held fixed; an interventional distribution still averages over exogenous realizations rather than retaining one episode's realized profile. Retaining the same realization is the additional cross-world commitment made by a counterfactual. This modularity clause is what makes the do-operator a useful model of action: when you set the thermostat, the model changes one rule and lets the rest of the causal network propagate the consequences.

The word "mutilates" is chosen deliberately and is worth a moment of discomfort. Deleting the edges into $X$ does not mean the parents stop influencing the world; it means they stop influencing $X$ through the intervened mechanism. In the sprinkler graph, $\mathrm{do}(T = 1)$ deletes the edge from $S$ to $T$, so sprinkler state no longer depends on or conveys information about season, but season still causes rain, and rain still wets the grass. The truncated factorization expresses this operation algebraically: delete one factor from the product and substitute the forced value wherever the variable appears as a parent. For continuous variables, read the factors as conditional densities.

The essential distinction is between conditioning and intervening:

• Conditioning on $X = x$ means selecting the subpopulation where $X$ happens to equal $x$. The parents of $X$ are still active, so their influence remains.
• Intervening on $X = x$ means forcing $X$ to equal $x$ regardless of its parents. The influence of the parents is severed.

A useful way to hold this distinction in mind is to think about who is responsible for $X$ taking the value $x$. Under conditioning, the value arises through the normal operation of the mechanisms, so observing $X = x$ is also evidence about the parents: if the sprinkler is on, the season is probably dry. Under intervention, the value is imposed from outside, so it carries no evidence about anything upstream. Conditioning is listening; intervening is speaking. Confusing the two is a recurring source of error across applied fields, but selection bias, collider bias, regression to the mean, and off-policy distribution shift are distinct phenomena rather than synonyms for one mistake.

In the sprinkler model with $W = R \lor T$, both $P(\text{wet} \mid T = 1)$ and $P(\text{wet} \mid \mathrm{do}(T = 1))$ equal exactly 1, since a forced-on sprinkler wets the grass regardless of season. The season-mixing contrast instead appears at $T = 0$: observing that the sprinkler is off is evidence for the wet season and raises the probability of rain, while forcing it off carries no such evidence, so $P(\text{wet} \mid T = 0) \approx 0.51$ exceeds $P(\text{wet} \mid \mathrm{do}(T = 0)) = 0.45$. In this particular example, conditioning on $T = 0$ makes wetness look higher than intervention does, because a switched-off sprinkler is a marker for the wet season, and the two effects pull in opposite directions. The general lesson is that the sign and size of the gap between seeing and doing is unpredictable without a causal model; there is no rule of thumb like "association overestimates causation." The gap can go either way, which is why it must be reasoned about explicitly.

For agents, planning treats a chosen action as an intervention on the action node. Logged trajectories are different: their actions are assignments generated by a behavior policy. On-policy and off-policy data can differ in support without either being hidden-confounded, and behavioral cloning reproduces the behavior policy's conditional action distribution rather than automatically inheriting unmeasured confounding. Hidden confounding arises when information omitted from the modeled history influences both policy choices and downstream outcomes. Keeping policy assignment, support shift, and intervention semantics separate is essential when deciding what a fitted model can support.

Interventions come in flavors worth distinguishing, and the distinctions matter because real agents rarely perform the textbook version:

• Hard interventions replace the mechanism entirely, as in $\mathrm{do}(X = x)$.
• Soft interventions modify a mechanism without fully specifying the value, for example by randomizing $X$ or by altering the strength of a parent's influence while keeping noise. They can be more realistic for some systems and, when the intervention targets, design, availability, and model assumptions are sufficiently informative, can identify causal structure that the available hard-intervention design does not. A soft intervention that changes how $X$ depends on its parents can reveal direction in suitable model classes even when the experimenter cannot directly set $X$ to an exact value.
• Uncertain interventions arise when the realized target or strength of a perturbation is not known exactly. A robot command can be a hard intervention on a command variable while the physical actuator state remains a stochastic downstream consequence shaped by dynamics, latency, and contact. Modeling error appears when those two variables are collapsed; representing both preserves an exact command intervention without pretending the realized effect is certain.

Identification is the question of when a Rung 2 quantity can be computed from Rung 1 data plus assumptions. This framing converts a philosophical question, "can we ever learn causation from correlation?", into a precise, per-model question: "given this graph, observed variables, and distributional assumptions, is the interventional quantity determined by the observational distribution?" Sometimes the answer is yes and sometimes no. One standard tool is the backdoor criterion: if $Z$ contains no descendant of $X$ and blocks every path from $X$ to $Y$ that starts with an arrow into $X$, then, under consistency and positivity,

where:

• $Z$: the adjustment set, the observed variables that block every backdoor path from $X$ to $Y$
• $z$: a value the adjustment set can take; the sum averages over all of them (replace it by an integral when $Z$ is continuous)
• $P(Z = z)$: how often that value occurs, which weights the contribution of each stratum

The intuition behind the formula is worth spelling out. A backdoor path is a route of association between $X$ and $Y$ that does not flow through the causal arrow $X \to Y$, typically a common cause like the heat load in our opening example. Under the stated assumptions, stratifying on $Z$ compares units with the same measured common causes; the outer sum then averages the stratum-specific conditional outcomes over the target population. This can identify the same population contrast as a randomized experiment, though it does not literally randomize the data and remains vulnerable to unmeasured confounding and model error. Off-policy importance weighting is related but different: state or context supplies the adjustment strata, while weights such as $\pi_{\mathrm{target}}(a\mid s)/\pi_{\mathrm{behavior}}(a\mid s)$ correct the behavior policy's action selection.

This is the adjustment formula. It tells you that with a valid adjustment set and the required support, an interventional distribution can be computed from observational conditionals. When no valid observed set exists, the effect is not identified by this formula; an experiment, an instrument, a front-door strategy, a proxy model, or other stronger structural assumptions may still identify it. The underlying limitation is informational rather than a matter of estimator sophistication. Two causal models can assign the same probability to every observed dataset while disagreeing about the effect of $X$ on $Y$. In off-policy evaluation, propensity weighting corrects policy selection only when the needed exchangeability and support assumptions hold.

An interventional distribution describes outcomes under a specified intervention and may itself be conditional on history. A counterfactual additionally conditions on evidence from the realized episode and reuses its exogenous factors under an alternative action. In the recursive, uniquely solvable SCMs used here, fixing those factors defines one outcome for each hypothetical intervention; abduction generally leaves posterior uncertainty over which factors occurred.

It is instructive to see how the three steps relate to the rungs below them. Step 2 and step 3 together are a Rung 2 computation, an intervention evaluated in the modified model. The additional ingredient is step 1, abduction, which conditions on the episode's evidence and updates a posterior over its exogenous factors rather than necessarily identifying them exactly. In this sense counterfactuals require what an intervention requires plus a model of how the observed episode constrains the unmodeled factors at hand.

The exogenous variables encode episode-specific background factors: the hidden friction in a particular joint, the heat load in a particular room, or an unmodeled gust. Abduction updates a posterior over those factors from what was observed, and prediction reuses the same exogenous realization in the alternate history. This is why counterfactuals are Rung 3: they require a specific cross-world pairing. Observational and interventional distributions need not determine that pairing without additional structural assumptions, although counterfactuals can be identified in some models when those assumptions and suitable data are available. Two episodes with identical observable histories may still have different exogenous factors and hence different counterfactual outcomes.

For an agent, Rung 3 appears when the question explicitly asks what would have happened in the same episode under a different choice. Diagnosis and blame assignment often have this form. Standard advantage functions are related decision comparisons but usually contrast conditional expected returns, not unit-level cross-world outcomes. Hindsight experience replay is different again: it relabels an achieved goal on a realized transition when the transition dynamics do not depend on the goal. Neither method becomes a Pearl-style counterfactual unless an SCM specifies how the same exogenous disturbances are coupled across actions.

The price of this power is strong identifiability requirements. Counterfactuals depend on the functional form of the mechanisms and the distribution of exogenous noise; the graph alone does not determine them. Two SCMs can induce identical observational and interventional distributions yet disagree on counterfactuals. Interventional and some counterfactual distributions can nevertheless be identified from graphical assumptions together with suitable observational or experimental distributions; counterfactual-identification results characterize important such cases. When you build counterfactual reasoning into an agent, you are making stronger structural commitments than ordinary prediction requires, so learned counterfactual outputs should be reported with their assumptions and uncertainty rather than treated as ground truth.

Because $C$ points into both $X$ and $Y$, variation in $C$ can create association along a backdoor path unrelated to the effect of $X$ on $Y$. The observational distribution alone need not reveal how much association comes from each path. In the linear additive worked example, covariance admits a direct closed-form decomposition; that two-term arithmetic is a property of the example, not of arbitrary nonlinear systems.

Confounding is one important way learned world models can be wrong in a systematic and exploitable way. In our opening example, the hidden heat load causes both the operators' cooling decisions and the temperature, so the learned association mixes the load's effect with cooling's effect. Unlike zero-mean sampling noise, this bias persists in the population regression and can give a planner a wrong gradient. Ensembles or pessimistic objectives may help when their uncertainty model spans the misspecification, but they do not automatically detect a shared confounding bias. Its size also depends on the behavior policy and action-noise distribution rather than being uniformly largest in the densest region.

Three facts about confounding are worth internalizing:

• It is not generally identifiable from one observational distribution alone. Distinct causal models can induce the same distribution of $(X,Y)$ while disagreeing about the effect of $X$ on $Y$. Distinguishing them requires assumptions, additional measured variables, multiple environments, temporal information, or interventional probes. In special model classes, observational constraints can still orient some edges, so the limitation should be read as non-identifiability in general rather than impossibility in every case.
• It can arise from insufficient recorded state. If a representation omits a common cause of action and outcome, adding a valid measured adjustment variable can close that backdoor. State insufficiency is broader: it can harm prediction or control without creating confounding, and causal identification may also fail through positivity, selection, or measurement error. Instruments, proxies, experiments, partial-identification bounds, and sensitivity analysis provide alternatives when the common cause cannot simply be measured.
• Interaction can remove or introduce selection dependence. Randomizing an action independently of relevant common causes breaks the behavior-policy backdoor for that randomized component. Merely adding stochasticity, including epsilon-greedy exploration, does not guarantee this if randomization probabilities still depend on hidden state or if support remains poor. A deterministic policy based on hidden state can create severe confounding, but its magnitude depends on the structural equations and noise. Designing safe, informative action randomization is therefore an experimental-design problem, not a universal dial with a guaranteed monotonic benefit.

If mechanisms are modular and independent, then a causal model should be stable under changes that do not touch its mechanisms. This stability creates an opportunity, because it converts distribution shift from a problem into a signal. If we can observe the world under several different input distributions, the relationships that persist across all of them are candidates for mechanisms, and the ones that break are candidates for artifacts. Two research programs turn this intuition into learning algorithms, and both matter enormously for world models.

Invariant causal prediction attacks the problem from the angle of distribution shift. Suppose you have data from several environments and a target variable $Y$. Its guarantee is conditional on assumptions about the interventions, target mechanism, model class, and statistical test:

• Under the method's assumptions, a predictor based on the target's causal parents has an invariant conditional distribution across environments that do not intervene directly on the target mechanism.
• A noncausal predictor is rejected only when the available environments shift the relevant dependence enough to make its conditional residual distribution change; an uninformative collection of environments may leave a spurious predictor invariant.

The logic depends on the environments doing useful work. If a hidden common cause changes its distribution, the induced association may change and expose a predictor that absorbed it. If that cause never shifts, or shifts in a cancelling way, the same spurious relation can survive every observed environment. Residual invariance is therefore a test relative to a specified family of environments and assumptions about which mechanisms were perturbed. Passing makes a predictor a causal candidate; it is not proof that nothing environmental leaks into the fit.

A noncausal feature may be fragile under a shift that perturbs the dependence it exploits, while a causal predictor remains invariant only across environments that leave its target mechanism and measurement process unchanged. For a world model, residual invariance can be a useful training or testing signal when environments correspond to a specified intervention design. Arbitrary partitions and penalties do not inherit ICP guarantees and can reject valid heteroskedastic mechanisms, so environment choice, target-mechanism assumptions, support, and test power must be documented.

The residual-invariance idea can be demonstrated with the climate system itself. We simulate the same building under two operator policies that react differently to the heat load while holding the heat-load and outcome-noise distributions fixed. In this deliberately controlled comparison, the confounded per-environment fits produce a larger residual shift than the interventional mechanism fit. That result illustrates the diagnostic; it is not a theorem that every confounded fit must fail every available invariance test.

Transportability asks the complementary question: you have a causal model learned in a source domain and want to predict in a target domain where some mechanisms may differ. Selection diagrams encode assumptions about which population-specific mechanisms differ. Graphical separation and do-calculus then determine whether a target interventional quantity can be expressed using the available source and target distributions. This is more selective than declaring the entire source model invalid, but it is not simple reachability from a changed node, nor does the diagram alone supply the required distributions.

For world models, transportability is the formal skeleton under several practical phenomena:

• Sim-to-real gap. A simulator and its target system may differ in contact dynamics, friction, sensor noise, actuator latency, or other mechanisms. If the differences are correctly localized, autonomous, and identifiable from sufficient target data, a modular model can reduce the scope of re-estimation. A monolithic approximator can also use frozen layers, adapters, or regularization; what it lacks is an explicit causal guarantee that a particular parameter subset corresponds to the changed mechanism.
• Nonstationarity in the wild. Furniture moves and user preferences drift. If a change is localized, its target is identified, non-target mechanisms are autonomous, and target data provide support, the agent can re-estimate a module while retaining justified prior knowledge elsewhere. Without those conditions, a local-update interpretation is a hypothesis to test rather than a guaranteed cure for forgetting.
• Meta-learning across tasks. When tasks share identifiable autonomous mechanisms, those mechanisms are candidates for reusable knowledge. Other transferable structure can also matter, and meta-learning can fail for optimization, capacity, support, or task-distribution reasons unrelated to causal modularity.

Everything above assumed we know the variables. A robot with a camera does not. It gets pixels, and the questions become: what should the latent state be so that causal reasoning over latents is valid? Can we learn latent variables that correspond to stable mechanisms, so that interventions on the world correspond to interventions on specific latents?

The stakes of this question are easy to state. Every result in this chapter, every adjustment formula, every transportability computation, every abduction step, presupposes a vocabulary of variables with the right causal relationships among them. If the latents are entangled, so that one latent coordinate mixes the heat load with the cooling effort and another mixes both with the time of day, then an intervention on the world moves several latents at once, and a single-coordinate edit such as $\mathrm{do}(Z_j=z)$ need not correspond to a realizable, localized physical intervention. The formal machinery can then answer a precisely posed question about the learned variables while that question fails to match the physical action the planner intended.

This is the program of causal representation learning, crystallized by Schölkopf and colleagues (2021). It combines the latent-variable methods of Part III with the causal machinery of this chapter, and its central result is sobering first, hopeful second.

First, the sobering part. Unrestricted nonlinear independent component analysis is not identifiable from i.i.d. observations alone: observationally equivalent nonlinear reparameterizations can yield incompatible latent coordinates when the decoder and latent law transform with them. An equally predictive representation can therefore remain an entangled mixture of causal variables. Temporal, multi-view, nonstationary, or other auxiliary structure can change that conclusion, so the impossibility belongs to the unrestricted observational setting rather than to every objective that lacks explicit interventions.

Second, the hopeful part. Identifiability results become possible when a particular model class is paired with sufficient auxiliary structure and assumptions. Embodied interaction can supply some of that structure, but none of the following ingredients is sufficient by itself:

• Temporal and intervention structure. Slow or sparse transitions can be useful assumptions in some domains, but they are not automatic properties of sequential data. CITRIS, for example, uses temporal intervened sequences together with observed intervention-target indicators and assumptions including invertible observations, stationarity, and no instantaneous effects. Its identification result does not follow from transition sparsity alone.
• Interventions and actions. Deliberately varied, well-characterized actions can provide interventional supervision when targets are known, interventions are sufficiently diverse, and the observation and dynamics assumptions support identification. Ordinary curiosity-driven exploration does not guarantee these conditions: actions may affect multiple unknown targets, be mediated by hidden state, or leave important factors untouched. Interaction is therefore a promising data source, not identifiability for free.
• Controllability structure. In some domains it is useful to distinguish factors the available actions can influence from nuisance factors they cannot. This is a modeling decomposition rather than a universal content-style split. When it is justified, training can encourage action-conditioned dynamics to represent controllable factors without treating every source of visual variation as a planning variable.
• Sparsity and modularity priors. Assuming that each mechanism depends on few variables can improve identifiability in specified model classes. Object-centric representations, covered in Object-Centric and Relational Worlds, provide useful candidates for groups of causal variables, but objects are not automatically mechanisms and an object-centric representation does not inherit causal modularity without further evidence.

For a world model builder, an ordinary i.i.d. predictive loss alone does not guarantee causal latents. Actions, time, multiple environments, or auxiliary variables can add the structure needed by a particular identification result. Action-conditioned latent dynamics can still support planning without coordinate-wise disentanglement when planners optimize realizable action sequences rather than editing latent coordinates directly. The stronger causal-latent goal remains valuable for interpretable interventions, transfer, and diagnosis, but it is a method-specific guarantee rather than a universal prerequisite for predictive-embedding or JEPA-style planning.

Let us now make the opening story precise and small enough to compute by hand. The system has three variables:

• $U = \eta_U$ with $\eta_U \sim \mathcal{N}(0, 1)$: a latent endogenous root representing the day's hidden heat load (sunlight, occupancy), unobserved by the modeler. This scalar $U$ is distinct from the formal exogenous set $\mathcal{U}$ above.
• $A$: the cooling effort. In the historical logs, operators react to the heat: $A = 0.8\,U + \eta_A$ with $\eta_A \sim \mathcal{N}(0, 0.5^2)$.
• $Y$: the room temperature deviation from comfort, with a baseline of 2 degrees: $Y = -1.5\,A + U + 2 + \eta_Y$ with $\eta_Y \sim \mathcal{N}(0, 1)$.

For the derivations below, $U$, $\eta_A$, and $\eta_Y$ are centered and mutually independent.

Before computing anything, trace the causal graph. $U$ points into both $A$ and $Y$, forming the backdoor path $A \leftarrow U \to Y$, while the direct causal path $A \to Y$ carries the true mechanism with slope $-1.5$. The behavior policy makes $A$ an increasing function of $U$, so hot days get more cooling; the mechanism makes $U$ push $Y$ up. Both effects of $U$ therefore push the observational association between $A$ and $Y$ in the same weakening direction, which is why the bias below is so large.

The bias formula rewards one more look before we simulate. Because the bias term $\alpha \cdot \text{U\_VAR} / (\alpha^2 \cdot \text{U\_VAR} + \text{SIGMA\_A}^2)$, which here simplifies to $\alpha / (\alpha^2 + 0.25)$, depends on how strongly operators react to the heat load and how much independent action noise they inject, we can sweep the reaction strength and watch the fitted slope travel between the causal value and a badly flattened one at intermediate reaction strengths.

We will simulate this SCM, fit models on observational and interventional data, compute a counterfactual, and evaluate both planners in the true system. The examples use NumPy for computation and Matplotlib with the book's plotting helpers for figures, and they run on a CPU in seconds. The progression gives numerical counterparts for the chapter's central worked claims: ordinary least squares produces a biased observational slope and approximately recovers the causal slope from randomized data, while the same-episode calculation and planner comparison expose the downstream consequences.

Now we define the SCM and sample a historical log. The heat load $U$ is drawn but kept "hidden" from the modeler: we will only pass it to the data-generating process, never to the naive model. This discipline, keeping the hidden variable present in the generator and absent from every estimator, is what makes the experiment an honest simulation of partial observability rather than a rigged comparison.

The bar chart quantifies the penalty: the naive plan's true cost is roughly four times the causal plan's, even though the naive model assigned its own plan a near-minimal predicted cost. The asymmetry between the two bars is the chapter's final lesson in numerical form. The difference in cost is not driven by noise, which both plans share equally through the $\text{Var}(U) + \text{SIGMA}_Y^2$ term in the expected cost; it is driven entirely by the squared bias term, the distance between each plan's mean temperature and the target. Bias is what confounding buys you, and squared bias is what it costs.

The key parameters for the confounded climate controller example are:

• CAUSAL_SLOPE: The true effect of cooling on temperature, $-1.5$ degrees per unit of effort. This is the mechanism quantity that the naive regression fails to recover. It is also the quantity that is invariant across all the questions in the example: the interventional fit recovers it, the counterfactual computation uses it, and the true expected-cost function is built from it.
• CONFOUNDING: The strength of the operators' reaction to the hidden heat load, $0.8$. It strengthens action-load coupling, but the slope bias is nonmonotonic: for these parameters it peaks at $0.5$ and then shrinks. At $\text{CONFOUNDING} = 0$, the backdoor closes and the observational regression is unbiased.
• SIGMA_A: The standard deviation of the noise in operator behavior, $0.5$. It controls how much cooling variation comes from the confounder rather than from deliberate variation of the action. The ratio $\text{CONFOUNDING}^2 / (\text{CONFOUNDING}^2 + \text{SIGMA}_A^2) \approx 0.72$ is the fraction of action variance driven by the hidden heat load, but the slope bias itself is $\text{CONFOUNDING} \cdot \text{U\_VAR} / (\text{CONFOUNDING}^2 \cdot \text{U\_VAR} + \text{SIGMA}\_A^2) \approx 0.90$; the bias peaks at intermediate reaction strengths and shrinks for very strong reactions rather than scaling monotonically with that fraction.
• SIGMA_Y: The standard deviation of the temperature noise, $1.0$. It contributes to outcome variance and expected cost but not to omitted-variable bias. The two designs can still have different finite-sample slope precision because their action distributions differ.
• BASELINE: The baseline temperature deviation from comfort, $2.0$ degrees, which sets the cooling level both planners must apply. Without it, the optimal action would be zero and the planning contrast would vanish, so this parameter is what gives the planning comparison its teeth.
• U_VAR: The prior variance of the hidden heat load, $1.0$, used by the abduction step and by the true expected-cost computation. In the abduction it acts as a regularizer, pulling the inferred load toward zero when the evidence is weak; in the cost function it appears as irreducible variance that no plan can remove.
• n_obs, n_int: The observational and randomized samples each contain $4000$ points. Equal nominal counts sharpen the population-bias comparison, although action support and independent noise realizations still differ across the designs.

It would be easy to leave this chapter with the impression that the fix is always "adjust for the confounder" or "run the randomized experiment." Adjustment requires observing a valid set with support; in partially observed settings, relevant common causes may be unavailable. Conditioning on a collider or its descendant can instead induce selection bias, including when outcome-based filtering selects through such a variable. Randomization can be expensive or unsafe and identifies only the assignments it randomizes with adequate support. Counterfactual queries make stronger structural commitments than interventional ones. In high-dimensional neural settings, both discovery and representation-learning guarantees remain method- and assumption-specific, with computational and statistical difficulty depending on the graph, model class, interventions, and observation process.

To these structural limits, add a practical one about the granularity of mechanisms. The theory is silent on where to place the module boundaries: whether contact physics is one mechanism or a family of per-object mechanisms, whether a user's preference is one variable or a time-varying hierarchy of them. Different granularities support different interventions and different transportability conclusions, and choosing among them is a modeling judgment that the formalism constrains but does not determine. This is the same tradeoff space we faced in building state abstractions and latent models, and the honest summary is that causal thinking sharpens the questions without mechanizing the answers.

The causal viewpoint changes a model's semantics, not necessarily its software shape. Predictive and causal models can both be modular neural systems; the causal model additionally states how variables and mechanisms respond to interventions. When changed mechanisms are localized, autonomous, and identifiable, that structure can justify selective re-estimation. Hidden confounding is one possible reason offline dynamics fail under new policies, while limited support and approximation error are others. Interventional data, valid adjustment, invariance across informative environments, and robust bounds are possible tools rather than an exhaustive cure list.

It also separates four useful notions of quality. Predictive fidelity describes fit to a specified distribution. Disentangled representations can make intervention targets interpretable, though an invertible entangled state plus a correct action-conditioned model may still support some causal queries. Uncertainty quality is relative to a query distribution, and in-distribution calibration does not establish interventional calibration. Decision usefulness depends on the task: causal structure is especially valuable when planning changes the data-generating process, but useful decisions do not logically require an explicit causal representation in every setting.

For evaluation, test the model under the queries it will receive: held-out prediction under the logging distribution, randomized or scripted actions that test interventional validity, and controlled mechanism changes that test localized adaptation. A discrepancy under intervention may reflect confounding, limited support, or approximation error, so the evaluation should be designed to separate them. Counterfactual probes are useful only where a simulator or other trusted construction supplies the same-unit ground truth.

• Causal models add intervention semantics to distributions. A structural causal model assigns variables generating rules. Under an explicit autonomy assumption, non-target rules remain fixed for the contemplated intervention; useful transfer requires that assumption to hold and the changed mechanism to be identifiable.
• Seeing is not doing. $P(Y \mid X)$ and $P(Y \mid \mathrm{do}(X))$ can differ because interventions replace the assignment mechanism. Logged models are confounded when omitted causes influence both policy assignment and outcomes; policy-generated data are not automatically confounded when sufficient history, exchangeability, and support hold.
• Counterfactuals need structure, not just statistics. Abduction–action–prediction supports per-episode "what if" reasoning, but only under strong assumptions about mechanisms and noise that distributions alone cannot certify. Two models can agree on everything observable, including all interventions, and still disagree on counterfactuals, so counterfactual outputs from learned models should be treated as diagnostics, not ground truth.
• Insufficient recorded state can create hidden confounding, but the concepts are not identical. Adjustment, experiments, instruments, proxies, sensitivity analysis, and partial-identification bounds address different failure modes. Conventional uncertainty estimates that omit confounding may fail, while sensitivity-aware or robust planners can sometimes act from bounded effects.
• Invariance and transportability support conditional transfer claims. An invariant relation is a causal candidate only relative to informative environments and target-mechanism assumptions. Localized sim-to-real repair further requires correct localization, autonomy, identifiability, and sufficient target data.
• Causal representation learning needs extra structure. Temporal processes, controlled interventions, multiple environments, auxiliary variables, and scientific experiments can all supply it. Action-conditioned latent dynamics may support planning over realizable actions without coordinate-wise causal latents; direct latent editing requires stronger intervention semantics.

The next chapter develops structured simulators, where some interventions can be implemented as edits to an explicit computational graph, alongside neural operators, which learn mappings between function spaces and are not intrinsically causal.

Ready to test your understanding? Take this quick quiz to reinforce what you've learned about causal models, interventions, and why seeing is not doing.