The World-Model Evaluation Ladder › Counterfactual intervention

The intervention rung asks how predictions change when the action protocol changes. A controlled simulator comparison and an observationally identified causal effect are different ways of obtaining an answer. Predictive accuracy on collected transitions is not enough to establish the latter.

For a deterministic environment $F$, fix an initial state $s_0$ and two action sequences, baseline $a_{0:H-1}$ and intervention $b_{0:H-1}$. Run both sequences from that common state in the true and model systems. The final response is

The response error is $\|\Delta_H^{\mathrm{model}}-\Delta_H^{\mathrm{true}}\|$. Superscripts identify the action branch; the two branches share $s_0$. Changing only actions isolates their effect within this deterministic experiment. If the two model branch errors have an equal additive component, that component cancels in the response difference. Unequal or action-dependent errors need not cancel.

In a stochastic system, a transition kernel specifies each branch's marginal law, not how their exogenous randomness is paired. A unit-specific counterfactual comparison needs that additional causal structure. For example, two branches can each be Bernoulli$(1/2)$ while sharing the same outcome, taking complementary outcomes, or using independent outcomes. Their marginal means coincide, but their paired differences have different distributions. A structural transition $s_{t+1}=F(s_t,a_t,U_t)$ can specify a shared sequence $U_t$ across the branches; a kernel alone does not supply it. Pearl's structural-counterfactual account makes this dependence on background variables explicit.

Alternatively, define an average intervention response as $\mathbb E[s_H^b]-\mathbb E[s_H^a]$. That difference of marginal expectations does not require a cross-branch noise coupling, though estimating it from observational data still requires identification assumptions. Neither definition should be silently substituted for the other.

For nonparametric adjustment-based observational identification, specify a causally adequate pre-action state, consistency, conditional exchangeability (no unmeasured action-outcome confounding), and positivity for the actions being compared. Multi-step policies need corresponding sequential conditions. State-dependent action selection is not automatically confounding after conditioning on an adequate state. A deterministic behavior policy can nevertheless leave alternative actions unsupported. Extrapolating their effects requires additional structural assumptions. The roles of consistency, exchangeability, and positivity are developed in Hernán and Robins' Causal Inference: What If.

Two practical distinctions help interpret the score.

• Data and assumptions: unsupported action effects cannot be recovered from observational accuracy alone. Additional experiments, structural knowledge, or scientific priors can constrain them. A poor response result may reflect data coverage, incorrect assumptions, or model error; changing architecture alone does not establish identification. [Part IV, Ch 3: Causal Models and Interventions] treats these choices.
• Response features: sign, magnitude, and timing can matter differently to a controller. A final-state norm does not separately diagnose all three. Report the response itself alongside a scalar error when these distinctions matter.

A successful response test covers the specified states and interventions. It does not guarantee useful optimization gradients, safe control, or accuracy on all alternative actions. Those need further tests.