Locomotion, Humanoids, and Navigation › Cross-Embodiment and Sim-to-Real Transfer
World Models HandbookAll chapters Locomotion, Humanoids, and Navigation › Cross-Embodiment and Sim-to-Real TransferOne section · Sections in this chapter5 / 8 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
Up to this point we have assumed a single robot and a single terrain. Real robots differ in mass, actuators, wear, and payload, and collecting failure-rich locomotion data on hardware can be costly or damaging. Simulation is therefore an important training and testing tool, raising the question of how a model transfers to physical hardware or another body.
Cross-embodiment is the study of world models and policies that transfer across morphologies: a quadruped that can be re-trained to walk on a different quadruped, a humanoid that shares a world model with a small biped, a manipulation policy that generalizes across gripper widths. Sim-to-real transfer moves a model or policy from simulation to physical hardware; it may keep the same embodiment or involve an embodiment change.
Cross-embodiment is plausible because robots share conservation laws and contact mechanics, even though their kinematics, actuators, observation spaces, and feasible contacts differ. A transferable representation can encode shared structure while retaining body-specific state and action interfaces. Whether joint training helps is an empirical question for the bodies and tasks involved.
Why Sim-to-Real is Hard
Three gaps bite.
The dynamics gap is mismatch between simulated dynamics and the physical robot. Real joints may exhibit friction, backlash, stiction, saturation, thermal effects, and delay that a particular simulator approximates imperfectly. Mass and actuator parameters may also be uncertain. These errors can alter contact timing or forces, sometimes changing a stable predicted step into a stumble; the size of the effect depends on the robot, controller, and surface.
The perception gap is mismatch between simulated and real sensing. A simulator may omit depth holes, reflections, lighting changes, lens distortion, latency, or other noise; these effects can also be modeled or randomized. The importance of a particular gap depends on which sensors the controller uses and at what rates. Exteroceptive terrain estimates can be especially sensitive to visibility and surface appearance, but no fixed slow-loop architecture follows.
Contact modeling can be a substantial source of sim-to-real mismatch. Contact geometry, friction, compliance, and numerical handling of impact can differ from physical behavior, and their effects interact with actuation and sensing. This chapter does not measure whether contact is the largest gap for any particular robot; that comparison requires target-system tests.
What Makes Transfer Work
We can state one transfer objective as follows: train a fixed policy on a source distribution over environment parameters and evaluate its return under a target distribution . We want to remain useful. The difference between source and target expectations depends on both the distribution shift and how the fixed policy's return varies with ; coverage alone does not establish target performance.
We can see the coverage argument directly. On a single environment-parameter axis, a narrow source concentrates its mass far from the target, while a wide source spreads its probability mass across a broader range that includes the target.
## Synthetic source and target densities over one environment parameter.
## The narrow source is a single Gaussian; the wide source is a three-component
## mixture; the target is a Gaussian near one end of the axis.
param_grid = np.linspace(0.0, 1.0, 200)
def gauss_pdf(x, mu, sigma):
return np.exp(-0.5 * ((x - mu) / sigma) ** 2) / (sigma * np.sqrt(2 * np.pi))
narrow_pdf = gauss_pdf(param_grid, 0.3, 0.05)
wide_pdf = (
0.33 * gauss_pdf(param_grid, 0.25, 0.08)
+ 0.34 * gauss_pdf(param_grid, 0.5, 0.1)
+ 0.33 * gauss_pdf(param_grid, 0.75, 0.08)
)
target_pdf = gauss_pdf(param_grid, 0.7, 0.06)
One technique is domain randomization: train on varied simulated dynamics and terrain so the policy encounters more of the conditions it may face later. Coverage of a target parameter range does not guarantee small transfer error, and finite data, mismatched observation and action interfaces, and optimization can still matter. Curriculum-style schedules may begin with a narrower range and widen it later, but their effect must be measured for the particular task and robot.
A second technique is system identification: use real observations to estimate parameters relevant to prediction or control, then adapt a simulator, model, or policy accordingly. Estimation may be performed offline or online. The architecture and measured benefit depend on the robot; this chapter does not establish a general actuator-calibration and terrain-adaptation pattern for Cheetah-style controllers.
A third is residual or adaptation learning: start from a model or policy trained under nominal conditions and learn a correction from target-domain data. The amount of real data and risk during adaptation depend on coverage, action limits, and validation. A nominally working policy does not by itself make online adaptation safe, and no broad industrial prevalence claim follows from this example.
A fourth is rapid adaptation: train a policy or estimator across varied conditions so it can use a short recent history, context vector, or a few update steps on a new condition. Terrain and payload changes can make this useful for locomotion, as demonstrated in named adaptation studies. Whether rapid adaptation outperforms a well-tuned fixed or robust controller depends on the target distribution and evaluation.
Cross-Embodiment Representation Learning
One approach to cross-embodiment modeling is to represent the robot as a graph of links and joints. Shared update rules can operate over different graph structures, while body-specific encoders, decoders, or action interfaces handle differing sensors and actuators. This is a design pattern, not a guarantee that a single learned model transfers across arbitrary morphologies.
The body-graph approach addresses one practical interface problem. Flat joint-angle vectors have different lengths for robots with different joint counts, so a fixed-width model needs padding, adapters, or another alignment scheme. A graph model instead shares a local update rule across links and joints of varying number. That gives it a mechanism for parameter sharing across body sizes, but generalization still depends on training coverage and physical similarity.
Training across many bodies could help a model learn reusable structure, but it also expands the data and validation problem. A systematic error in shared parameters can affect multiple morphologies, and aggregate performance can hide body-specific failures. Evaluation must therefore report results by body and task rather than only a pooled score.
As in Part X, Ch 2: Robotic Manipulation, transfer depends on which physical and sensing structure is shared. Locomotion changes foot geometry, gait, and contact forces; manipulation changes end effectors, objects, and grasp contacts. Neither domain is uniformly easier to transfer across, and methods from one require validation before reuse in the other.
Sim-to-Real as World-Model Transfer
We can recast the whole discussion as a world-model transfer problem. Suppose we have a source world model trained in simulation, and we want a target model that predicts the physical robot. The simplest transfer model assumes that the target dynamics differ from the source dynamics by an additive residual, which lets us write:
where:
- : the source world model, trained in simulation
- : the target world model, describing the physical robot
- : the residual dynamics not captured by the source model
- : the robot state (e.g., joint angles and velocities)
- : the control action
The additive form is a modeling assumption, not a law. It requires comparable state and action interfaces; even then, a learned residual may not extrapolate reliably across changes in friction, mass, or compliance. When robots have different state or action dimensions, a direct additive residual is undefined until their interfaces are aligned. Padding, adapters, shared task-space variables, or body-graph representations are possible alignments; a graph is not required.
These four transfer techniques address different parts of the transfer problem; not all require the additive residual model:
- Domain randomization varies simulated parameters to expose a policy or model to a range of conditions. It does not require the residual to have zero mean; its benefit must be measured on target conditions.
- System identification estimates physical dynamics parameters from real rollouts; a residual model may instead learn parameters of directly.
- Residual learning trains a neural network to approximate directly.
- Meta-learning trains a policy whose behavior depends on a context that is inferred from a few real rollouts.
These techniques can be combined. Domain randomization exposes the learner to anticipated variation, while residual learning or online estimation can correct systematic target mismatch observed on hardware. Neither contribution is automatic; the combination must be evaluated on the target robot and terrain.
A Minimal Domain Randomization Experiment
We close with a small experiment that makes these ideas concrete. We generate two training sets: one uses a single fixed terrain, while the other uses ten randomly drawn terrains with three episodes on each. We fit a linear model of the vertical hopper dynamics on each set, then evaluate on a held-out terrain absent from both training sets.
def make_terrain_random(rng):
a1 = rng.uniform(0.05, 0.15)
a2 = rng.uniform(0.02, 0.08)
w1 = rng.uniform(3.0, 6.0)
w2 = rng.uniform(1.0, 2.0)
def terrain_fn(x):
return a1 * np.sin(2 * np.pi * x / w1) + a2 * np.sin(2 * np.pi * x / w2)
return terrain_fn
def collect_data(terrain_fns, n_per_terrain, rng, steps=1500):
X, Y = [], []
for tfn in terrain_fns:
for _ in range(n_per_terrain):
vz0 = rng.uniform(3.0, 5.5)
z0 = rng.uniform(0.3, 0.6)
_, zs_i, vzs_i, _ = simulate_hopper(
x0=0.0, z0=z0, vz0=vz0, vx0=1.5, terrain_fn=tfn, steps=steps
)
X.append(
np.column_stack(
[zs_i[:-1], vzs_i[:-1], np.ones_like(zs_i[:-1])]
)
)
Y.append(
np.column_stack([zs_i[1:] - zs_i[:-1], vzs_i[1:] - vzs_i[:-1]])
)
return np.concatenate(X, axis=0), np.concatenate(Y, axis=0)
def fit_linear(X, Y):
theta, *_ = np.linalg.lstsq(X, Y, rcond=None)
return theta
## Narrow source: one terrain
narrow_source = [terrain_height]
## Wide source: 10 randomized terrains
wide_rng = np.random.default_rng(0)
wide_source = [make_terrain_random(wide_rng) for _ in range(10)]
## Held-out target terrain: a new randomized terrain
target_rng = np.random.default_rng(999)
target_terrain = make_terrain_random(target_rng)
data_rng = np.random.default_rng(7)
X_narrow, Y_narrow = collect_data(narrow_source, 30, data_rng)
X_wide, Y_wide = collect_data(wide_source, 3, data_rng)
theta_narrow = fit_linear(X_narrow, Y_narrow)
theta_wide = fit_linear(X_wide, Y_wide)
print(f"Narrow model trained on {X_narrow.shape[0]} transitions")
print(f"Wide model trained on {X_wide.shape[0]} transitions")The two training sets contain the same number of transitions: 30 trajectories on one fixed terrain versus three trajectories on each of ten fixed, randomly drawn terrains. This controls the sample count but not every other source of variation. Any observed advantage on the one held-out target is a result of this toy setup, not a general guarantee for domain randomization.
We now evaluate each model as an open-loop predictor on the held-out target terrain and compare the rollout error.
## Ground-truth trajectory on the target terrain
xs_t, zs_t, vzs_t, _ = simulate_hopper(
x0=0.0, z0=0.5, vz0=4.5, vx0=1.5, terrain_fn=target_terrain, steps=1500
)
def rollout_error(theta_model, z_true, v_true, H=500):
z_pred = np.zeros(H)
v_pred = np.zeros(H)
z_pred[0], v_pred[0] = z_true[0], v_true[0]
for t in range(1, H):
x_in = np.array([z_pred[t - 1], v_pred[t - 1], 1.0])
delta = x_in @ theta_model
z_pred[t] = z_pred[t - 1] + delta[0]
v_pred[t] = v_pred[t - 1] + delta[1]
return np.abs(z_pred - z_true[:H])
err_narrow = rollout_error(theta_narrow, zs_t, vzs_t)
err_wide = rollout_error(theta_wide, zs_t, vzs_t)
## Report the average error over the rollout for each model
mean_err_narrow = float(np.mean(err_narrow))
mean_err_wide = float(np.mean(err_wide))Mean rollout error, narrow-trained model: 0.3298 m Mean rollout error, wide-trained model: 0.2982 m
Finally, we visualize the error comparison across the rollout horizon.
In this seeded toy comparison, the wide-trained predictor has lower mean open-loop height error on the single held-out terrain. That is not a closed-loop control result or evidence that widening the source always helps. A simple distributional bound explains one consideration: if the fixed policy's return on an environment parameter lies between zero and , then
where:
- : return of the fixed policy in environment .
- and : source and target distributions over the same environment-parameter space.
- : an assumed finite upper bound on nonnegative return.
- : total variation distance, defined as over measurable sets .
This bound is loose and says nothing about whether a learned policy succeeds on the target. A wider source can reduce some distribution mismatch while also making learning harder or placing too little training mass on important target conditions. Measure target performance directly and distinguish open-loop model error from closed-loop return.
Domain randomization trains a policy or model across a chosen distribution of simulated parameters or observations to improve robustness to anticipated variation. Whether that distribution covers the deployment target is an empirical question.