Michael Brenndoerfer | Data & AI, Private Equity, Technology

Michael Brenndoerfer

State, Physics, Causality, and Memory Evaluation › Interventions, Counterfactuals, and Causal Fidelity

Michael BrenndoerferAugust 2, 202611 min read

World Models HandbookAll chapters

Part 11 of 12Evaluation and Understanding

Chapter 63State, Physics, Causality, and Memory Evaluation › Interventions, Counterfactuals, and Causal Fidelity

Reading section 5 of 8 · Interventions, Counterfactuals, and Causal Fidelity · 11 minSection 5 of 8 · 11 min

Section 5 of 8 · 11 min read

State, Physics, Causality, and Memory Evaluation › Interventions, Counterfactuals, and Causal FidelityOne section · Sections in this chapter5 / 8 ↓

Sections in this chapter8

Reading tools Definitions & inline links

Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.

Article links

Make inline references clickable

The occluded-ball question from the previous sections was observational: given what was seen, what happens next? The questions in this section are different. They are about what would happen under a change. The difference between "given what was seen, what happens next" and "what would have happened if we had done something different" is precisely the difference between prediction and causal inference, and no amount of observational data can answer the second kind of question without additional assumptions.

Three targets must be kept apart, because they are estimated differently, identify different quantities, and are routinely confused:

Three targets, three meanings

Observational prediction. In a state-observed setup, predict $s_{t+1}$ from $(s_t, a_t)$ on the distribution the data were collected under. Other observationally trained sequence models can use observation histories or latent representations as inputs and targets, rather than true simulator states.

Population intervention. Compare $\mathbb{E}[s_H \mid \mathrm{do}(a_\tau=a^\star)]$ and $\mathbb{E}[s_H \mid \mathrm{do}(a_\tau=a_0)]$ over a specified population. Identification needs a suitable design and assumptions: for example randomized action assignment, or an observational adjustment design with consistency, positivity, and conditional exchangeability. Absence of hidden confounding is not necessary for every possible identification strategy.

Unit-level counterfactual. For one specific unit, estimate what $s_H$ would have been under a different action. In the structural causal model (SCM) used here, hold that unit's initial state and background disturbance fixed across the hypothetical branches. That is a specified cross-world coupling, not a universal ranking of how difficult the three targets are to identify.

Observational fit does not identify any causal quantity without further assumptions. A model can fit the observational distribution to machine precision and still be wrong about the effect of an intervention, and the mechanism by which this happens is confounding.

Building a confounded toyLink Copied

The earlier simulator sampled actions independently of its state and transition, without a shared action/outcome disturbance. We now switch to a separate one-step structural causal model in which an unobserved disturbance $u_t$ influences both the action and the velocity change:

\begin{aligned} u_t &\sim \mathcal{N}(0, 1) \\ a_t &= -\kappa\,(x_t - 0.5) + u_t \\ v_{t+1} &= v_t + \big(a_t + c\,u_t\big)\,\Delta t \end{aligned}

where:

$u_t$ : the unobserved exogenous acceleration disturbance, with numerical values in m/s² drawn from a standard normal.
$a_t$ : the acceleration action in m/s² chosen by the state-feedback policy with gain $\kappa=2$ s $^{-2}$ ; the position reference $0.5$ is in metres.
$x_t$ : the position at step $t$ .
$v_{t+1}$ : the next velocity, driven by the action plus the disturbance coupling $c$ .
$\Delta t = 0.1$ s: the step size.

The action is chosen by a state-feedback policy, but the policy's exploration noise is the same $u_t$ that enters the dynamics with coupling $c$ . An engineer observing only $(x_t, a_t, v_t, v_{t+1})$ never sees $u_t$ , so the observational data are confounded by construction. This toy illustrates one possible shared-signal mechanism: the observational correlation between action and outcome mixes the direct effect of the action with the effect of the shared signal. It does not establish how often that mechanism occurs in deployed systems.

Now ask: what is the causal effect of the action on the one-step velocity change? Holding $u_t$ fixed and differentiating, the structural derivative is

\frac{\partial\, \Delta v_t}{\partial a_t} = \Delta t = 0.1\ \text{m/s per m/s}^2

where $\Delta v_t = v_{t+1} - v_t$ is the one-step velocity change, $a_t$ is the action, and $\Delta t = 0.1$ s is the step size. The derivative is $\Delta t$ because the structural equation is linear in the action with slope $\Delta t$ . This is the quantity we want to recover, and it is a structural quantity: it describes how the next velocity would change if we intervened on the action while holding everything else fixed.

That is the ground truth we want to recover. Now see what an observational regression gives. Rearranging the policy equation gives $u_t = a_t + \kappa (x_t - 0.5)$ . Substituting this into the dynamics,

\frac{\Delta v_t}{\Delta t} = a_t + c\,u_t = (1 + c)\,a_t + c\kappa\,(x_t - 0.5)

where $u_t = a_t + \kappa (x_t - 0.5)$ has been substituted. The coefficient on $a_t$ is $1 + c$ rather than $1$ , so the observational relationship overstates the structural action effect. The empirical analogue is a regression of the observed velocity change on the observed action and position.

In the noiseless structural relationship, the observational coefficient is $(1+c)\Delta t=0.2$ , twice the intervention slope $0.1$ . The measured regression below includes observation noise and fits nearly, not exactly, perfectly. A predictive coefficient need not be an intervention effect.

In[29]:

Code

KAPPA = 2.0  # policy gain
C_U = 1.0  # strength of the unobserved disturbance coupling
DT_C = 0.1  # s


def sample_units(rng, n):
    """One-step structural units with a policy that depends on the disturbance."""
    x0 = rng.uniform(0.2, 0.8, n)
    u = rng.normal(0.0, 1.0, n)
    a = -KAPPA * (x0 - 0.5) + u  # action depends on the unobserved u
    dv = (a + C_U * u) * DT_C  # true one-step velocity change
    return x0, u, a, dv

In[30]:

Code

rng_c = np.random.default_rng(7)
x_all, u_all, a_all, dv_all = sample_units(rng_c, 4000)
dv_meas = dv_all + rng_c.normal(
    0.0, 0.005, dv_all.size
)  # measurement standard deviation 0.005 m/s, not a relative percentage

fit_slice = slice(0, 2000)
held_slice = slice(2000, None)

## Observational regression of dv on (x - 0.5, a). No interventions were used.
X_fit = np.hstack(
    [
        np.ones((2000, 1)),
        (x_all[fit_slice] - 0.5)[:, None],
        a_all[fit_slice][:, None],
    ]
)
beta_obs = ridge_fit(X_fit, dv_meas[fit_slice][:, None], alpha=1e-8)
effect_obs = float(beta_obs[2, 0])

X_held = np.hstack(
    [
        np.ones((2000, 1)),
        (x_all[held_slice] - 0.5)[:, None],
        a_all[held_slice][:, None],
    ]
)
pred_held = X_held @ beta_obs
ss_res = float(np.sum((dv_meas[held_slice][:, None] - pred_held) ** 2))
ss_tot = float(np.sum((dv_meas[held_slice] - dv_meas[held_slice].mean()) ** 2))
r2_obs = 1.0 - ss_res / ss_tot

support_lo, support_hi = float(a_all.min()), float(a_all.max())

The held-out observational fit is $R^2\approx0.9993$ , while its fitted action coefficient is about $0.19994$ instead of the structural slope $0.1$ . Fit alone does not reveal the confounding in this example. The nearly perfect noisy fit should not be described as machine precision.

The paired counterfactual experimentLink Copied

Now we run a paired simulator intervention on the held-out units. For each unit:

Clone the unit's initial state $x_0$ and its exogenous disturbance $u$ .
Run it once with its natural action $a$ .
Run it again with the same state and the same $u$ , but with the action shifted by a fixed $\delta$ .
Subtract the two outcomes.

Because $u$ is shared, the difference isolates the direct effect of the action. Substituting the structural equation $\Delta v(a) = (a + c\,u)\,\Delta t$ into the two branches,

\begin{aligned} \Delta v_t(a + \delta) - \Delta v_t(a) &= \big((a + \delta) + c\,u\big)\,\Delta t - \big(a + c\,u\big)\,\Delta t && \text{(structural equation at each action)} \\ &= \big(a + \delta + c\,u - a - c\,u\big)\,\Delta t && \text{(expand)} \\ &= \delta\,\Delta t && \text{(the shared } c\,u \text{ cancels)} . \end{aligned}

For the noiseless structural outcomes, dividing by nonzero $\delta$ recovers $\Delta t=0.1$ exactly. The code then adds independent measurement noise to both branches and averages across units, so its estimate has sampling error. Shared structural $u$ is the simulator's explicit counterfactual coupling; passive video does not supply that coupling by itself.

In[31]:

Code

DELTA = 0.5

x_h = x_all[held_slice]
u_h = u_all[held_slice]
a_h = a_all[held_slice]

dv_base = (a_h + C_U * u_h) * DT_C
dv_shift = ((a_h + DELTA) + C_U * u_h) * DT_C

noise_base = rng_c.normal(0.0, 0.02, a_h.size)
noise_shift = rng_c.normal(0.0, 0.02, a_h.size)
effect_paired = float(
    np.mean((dv_shift + noise_shift - dv_base - noise_base) / DELTA)
)
effect_true = float(DT_C)

effect_rows = [
    ("observational regression", effect_obs),
    ("paired counterfactual", effect_paired),
    ("ground truth (simulator)", effect_true),
]

Out[32]:

Console

estimator                      effect of a on dv    relative error
observational regression                  0.1999            99.9%
paired counterfactual                     0.0996             0.4%
ground truth (simulator)                  0.1000             0.0%

observational fit on held-out natural data: R^2 = 0.9993
observed action extrema: [-3.703, 3.692] m/s^2
intervention magnitude delta = 0.50; fraction above observed maximum = 0.1%
no intervened data was used to fit the observational model.

The observational slope is about $0.19994$ , whereas the paired noisy estimate is about $0.09957$ versus structural truth $0.1$ . The latter has roughly $0.4\%$ relative error in this run. Pairing is sufficient for this demonstration, not necessary for every population effect: assigning independently sampled units to fixed actions by randomization can identify the difference of population mean outcomes when the intervention is well-defined and observed outcomes are consistent with their assigned actions. Here the two groups have the same disturbance distribution, so their mean structural contrast divided by the action difference is $\Delta t$ . This experiment evaluates an estimator using the simulator's equations; it does not benchmark a learned model's counterfactual predictions.

Out[33]:

Visualization

Bar chart comparing observational, paired counterfactual, and ground truth causal effect estimates. — One-step action-effect estimates. Observational regression has held-out R-squared about 0.9993 but gives a slope near 0.2 instead of 0.1. Sharing the structural disturbance cancels it in noiseless branch differences; independent measurement noise makes the paired average approximate, not exact. This paired design is sufficient here, not necessary for all population-effect identification.

What else to report alongside a causal claimLink Copied

A single effect estimate is not enough to make a causal claim legible. Report at least the following. Each item corresponds to a distinct way in which a reader might reasonably disagree with the claim, and each is a way of making the claim falsifiable rather than rhetorical.

The exact target. Average treatment effect, conditional effect at a specific state, or a unit-level counterfactual? These are different numbers. State which one, and state the unit of the estimate.
Intervention support and overlap. Distinguish population support from finite observed extrema and from practical data coverage. Here Gaussian disturbances give actions full real-line population support, but about $0.1\%$ of shifted held-out actions exceed the finite sample maximum. Being within the sample range does not establish dense coverage or validate a causal estimate.
Which interventions were observed during training. Here, none: the model was fitted purely observationally, and every intervention was held out. If a model was trained on interventions, say which, and reserve a distinct held-out intervention family for evaluation.
The feasibility of the intervention. Some interventions are not physically implementable in the source system. A simulated intervention on an unfeasible variable is a modelling exercise, not evidence about the world.
Confounding and observability. Write down which variables are observed, which are latent, and which one is assumed to be the exogenous driver. Our pairing stipulates a shared structural $u$ across the two branches. Passive observational data alone do not generally identify that cross-world coupling, even if a dataset measures a relevant disturbance.
Shared-noise coupling. A unit-level counterfactual needs an explicit joint model across branches. Changing that coupling can change paired differences and estimator variance. With fixed branch marginals, the population difference of expected outcomes is unchanged; do not confuse a unit counterfactual with an average effect.
Simulator-to-world limitations. Every result in this section lives inside a simulator whose structural equations we wrote. The same protocol applied to real video would require the structural model to be justified externally, and that justification is usually where the argument lives.

Invariance across environments is diagnostic evidence, not an identification argument by itself. A stable coefficient can coexist with confounding, including an unchanged confounding mechanism across the sampled environments. State which environments changed and which identification assumptions the design supports; coefficient stability alone does not establish causal fidelity.

Action conditioning does not guarantee causal validity. Our observational regression takes action as an input yet estimates the wrong intervention slope. A causal method must represent the specified intervention somehow, but it need not be an action-conditioned prediction network; identification can use a different suitable experimental or structural design.

Counterfactual prompting is a method, not a guarantee

Prompting a learned model to generate a counterfactual specifies a desired output, not an identification argument. Validate the target effect using an appropriate intervention design and state the structural assumptions. A paired simulator experiment is one option, not a universal requirement.