Causal Models and Interventions › Invariance, Transportability, and Learning Across Environments
World Models HandbookAll chapters Causal Models and Interventions › Invariance, Transportability, and Learning Across EnvironmentsOne section · Sections in this chapter7 / 14 ↓
If mechanisms are modular and independent, then a causal model should be stable under changes that do not touch its mechanisms. This stability creates an opportunity, because it converts distribution shift from a problem into a signal. If we can observe the world under several different input distributions, the relationships that persist across all of them are candidates for mechanisms, and the ones that break are candidates for artifacts. Two research programs turn this intuition into learning algorithms, and both matter enormously for world models.
Invariant causal prediction attacks the problem from the angle of distribution shift. Suppose you have data from several environments and a target variable . Its guarantee is conditional on assumptions about the interventions, target mechanism, model class, and statistical test:
- Under the method's assumptions, a predictor based on the target's causal parents has an invariant conditional distribution across environments that do not intervene directly on the target mechanism.
- A noncausal predictor is rejected only when the available environments shift the relevant dependence enough to make its conditional residual distribution change; an uninformative collection of environments may leave a spurious predictor invariant.
The logic depends on the environments doing useful work. If a hidden common cause changes its distribution, the induced association may change and expose a predictor that absorbed it. If that cause never shifts, or shifts in a cancelling way, the same spurious relation can survive every observed environment. Residual invariance is therefore a test relative to a specified family of environments and assumptions about which mechanisms were perturbed. Passing makes a predictor a causal candidate; it is not proof that nothing environmental leaks into the fit.
A noncausal feature may be fragile under a shift that perturbs the dependence it exploits, while a causal predictor remains invariant only across environments that leave its target mechanism and measurement process unchanged. For a world model, residual invariance can be a useful training or testing signal when environments correspond to a specified intervention design. Arbitrary partitions and penalties do not inherit ICP guarantees and can reject valid heteroskedastic mechanisms, so environment choice, target-mechanism assumptions, support, and test power must be documented.
The residual-invariance idea can be demonstrated with the climate system itself. We simulate the same building under two operator policies that react differently to the heat load while holding the heat-load and outcome-noise distributions fixed. In this deliberately controlled comparison, the confounded per-environment fits produce a larger residual shift than the interventional mechanism fit. That result illustrates the diagnostic; it is not a theorem that every confounded fit must fail every available invariance test.
import numpy as np
## Constants and the OLS helper, defined here because the Code Implementation
## section that introduces them appears later in the chapter.
CAUSAL_SLOPE = -1.5
CONFOUNDING = 0.8
SIGMA_A = 0.5
SIGMA_Y = 1.0
BASELINE = 2.0
U_VAR = 1.0
def fit_ols(x, y):
X = np.column_stack([x, np.ones_like(x)])
coef, *_ = np.linalg.lstsq(X, y, rcond=None)
return coef[0], coef[1]
## Interventional fit from randomized cooling, used by the mechanism residual.
env_rng = np.random.default_rng(23)
n_int = 4000
u_int = env_rng.normal(size=n_int)
a_int = env_rng.uniform(0.0, 3.0, size=n_int)
y_int = (
CAUSAL_SLOPE * a_int
+ u_int
+ BASELINE
+ SIGMA_Y * env_rng.normal(size=n_int)
)
slope_int, intercept_int = fit_ols(a_int, y_int)
## Two environments of the same SCM with different behavior policies and an
## identical heat-load distribution. One fixed RNG stream supplies both
## environments for reproducibility; there are no ties to break and no
## selection rules beyond fixed sample size.
n_env = 4000
def sample_env(alpha, u_scale, rng):
"""Environment with reaction strength alpha and heat-load scale u_scale."""
u = u_scale * rng.normal(size=n_env)
a = alpha * u + SIGMA_A * rng.normal(size=n_env)
y = CAUSAL_SLOPE * a + u + BASELINE + SIGMA_Y * rng.normal(size=n_env)
return a, y
## Environment A: strong reaction (like the historical log).
## Environment B: weaker reaction, same heat-load distribution.
a_env_a, y_env_a = sample_env(1.2, 1.0, env_rng)
a_env_b, y_env_b = sample_env(0.4, 1.0, env_rng)
## Naive per-environment fits absorb confounding; the interventional fit
## (slope_int and intercept_int, estimated above) is evaluated in both
## environments.
slope_naive_a, intercept_naive_a = fit_ols(a_env_a, y_env_a)
slope_naive_b, intercept_naive_b = fit_ols(a_env_b, y_env_b)
resid_naive = np.array(
[
np.std(y_env_a - (slope_naive_a * a_env_a + intercept_naive_a)),
np.std(y_env_b - (slope_naive_b * a_env_b + intercept_naive_b)),
]
)
resid_mech = np.array(
[
np.std(y_env_a - (slope_int * a_env_a + intercept_int)),
np.std(y_env_b - (slope_int * a_env_b + intercept_int)),
]
)
Transportability asks the complementary question: you have a causal model learned in a source domain and want to predict in a target domain where some mechanisms may differ. Selection diagrams encode assumptions about which population-specific mechanisms differ. Graphical separation and do-calculus then determine whether a target interventional quantity can be expressed using the available source and target distributions. This is more selective than declaring the entire source model invalid, but it is not simple reachability from a changed node, nor does the diagram alone supply the required distributions.
For world models, transportability is the formal skeleton under several practical phenomena:
- Sim-to-real gap. A simulator and its target system may differ in contact dynamics, friction, sensor noise, actuator latency, or other mechanisms. If the differences are correctly localized, autonomous, and identifiable from sufficient target data, a modular model can reduce the scope of re-estimation. A monolithic approximator can also use frozen layers, adapters, or regularization; what it lacks is an explicit causal guarantee that a particular parameter subset corresponds to the changed mechanism.
- Nonstationarity in the wild. Furniture moves and user preferences drift. If a change is localized, its target is identified, non-target mechanisms are autonomous, and target data provide support, the agent can re-estimate a module while retaining justified prior knowledge elsewhere. Without those conditions, a local-update interpretation is a hypothesis to test rather than a guaranteed cure for forgetting.
- Meta-learning across tasks. When tasks share identifiable autonomous mechanisms, those mechanisms are candidates for reusable knowledge. Other transferable structure can also matter, and meta-learning can fail for optimization, capacity, support, or task-distribution reasons unrelated to causal modularity.