Data Curation and Synthetic Data › Simulation and Procedural Generation

Real data can be expensive or slow to collect, and safety constraints can make some outcomes rare. Synthetic data can target those gaps, with a fidelity cost. The term covers a wide spectrum, and the spectrum matters:

• Scripted simulation. A fixed scenario in a physics engine, replayed for many episodes.
• Domain randomization. The same scenario with scene layout, textures, lighting, mass, friction, latency, and actuator gains resampled every episode.
• Procedural generation. Layouts, object placements, and task configurations synthesized from a grammar or a sampler, producing combinatorially many distinct scenes.
• Learned generation. A generative model of trajectories, scenes, or videos, trained on real data, used to emit new samples.

The core idea in all four cases is the same: you trade physical fidelity for controllable coverage. A simulator with imperfect contact dynamics but perfect state labels and unlimited scenario variation may be far more useful for training a world model than a dataset of flawless real recordings that only ever show the robot succeeding. The trade only pays off when the simulator's imperfections are smaller than the gap it fills, which is why the ranking of "how useful is this simulator" depends entirely on where in the state space you are looking.

It helps to separate three distinct benefits, because they have different failure modes:

• Coverage expansion. Simulation can sample rare, dangerous, or expensive scenarios without executing them on hardware. These samples constrain the learned model only to the extent that the simulator is valid in those scenarios; targeted real tests remain the stronger evidence when they are feasible.
• Counterfactual branching. Starting from a simulator state, you can roll out several action sequences and obtain action-conditional transitions that were absent from the behavior-policy log. Branching from an estimated real state is only as valid as the state mapping and simulator. This complements the identifiability limits discussed earlier in Actions, Inverse Dynamics, and Latent Actions.
• Privileged labels. Simulators can expose internal state, poses, contacts, object identities, and unoccluded geometry without a perception pipeline. These labels are exact with respect to the simulator's own state, not necessarily with respect to the real system or the representation needed for control.

Simulation creates a distribution shift between $p_{\text{sim}}(s_{t+1} \mid s_t, a_t)$ and $p_{\text{real}}(s_{t+1} \mid s_t, a_t)$. If the training target were exactly the kernel mixture

$$p_\lambda=(1-\lambda)p_{\text{real}}+\lambda p_{\text{sim}},$$

at a fixed $(s_t,a_t)$, then total variation obeys the identity

$$D_{\mathrm{TV}}(p_\lambda,p_{\text{real}})=\lambda D_{\mathrm{TV}}(p_{\text{sim}},p_{\text{real}}).$$

This identity describes the idealized mixture distribution, not the error of a learned model. Finite samples, model misspecification, optimization, and different state-action marginals add other error terms.

Two consequences follow. First, simulated data is most valuable where real data has no support, because there the alternative is zero coverage rather than a small bias. Second, adding simulated data to a well-covered real regime can actively hurt: you are injecting bias in exchange for density you did not need. The mixture weight should be high where the coverage gap is large and low where it is small, which is exactly the trade-off the next section formalizes. Practically, this means you should define coverage per region before you set any synthetic weight, so that the entire allocation step is a function of measurements rather than intuitions.

Another form of synthetic data is to roll out a learned world model and train on those results. Dyna architectures integrate learned models with planning and learning, as discussed in Model-Based Reinforcement Learning. Short rollouts limit how far each synthetic trajectory can drift, but they do not remove model bias.

Recursive training can fail when generated samples replace or overwhelm the original data. Shumailov and colleagues call the resulting loss of information about the original distribution model collapse and demonstrate tail loss for several model families. Collapse is not inevitable under every recursive protocol: retaining real anchors, accumulating data, using faithful generators, or changing the estimator can alter the outcome. For world models, the corresponding risk is that rollout errors and underrepresented regimes are fed back into later training rounds.

The following controlled illustration isolates one collapse mechanism: selection against tails. It starts from 4,000 samples from a Student-$t$ distribution with three degrees of freedom, fits a Gaussian, generates 4,500 candidates, retains only the central 95%, and repeats for ten rounds with seed 7. The plotted statistic is the empirical 99th percentile of $|x|$. The central-selection step, not maximum likelihood by itself, drives the monotone contraction; a Gaussian fit resampled without truncation would fluctuate around a stable variance in expectation.