Data Curation and Synthetic Data › Trajectory Quality and Diversity

A world model trains on trajectories $\tau = (o_1, a_1, o_2, a_2, \dots, o_T)$, where $o_t$ is the observation at time $t$, $a_t$ is the action at time $t$, $T$ is the trajectory length, and each adjacent triple $(o_t, a_t, o_{t+1})$ for $t=1,\dots,T-1$ is a transition. Two important properties of a trajectory collection are how trustworthy each trajectory is and how broadly the collection covers the situations the model will be asked about. They are distinct rather than strictly orthogonal: collection choices can affect both. A pristine recording of a robot waving its arm in place for an hour may have high signal quality and little behavioral diversity. A varied hour containing recoveries and falls may have broad behavioral coverage even if some transitions are corrupted. Dataset size alone therefore cannot summarize either quality or coverage.

Quality is not a scalar property of a file. It is a set of separate failure modes, and each one demands a different detector. It helps to think of each failure mode as a specific corruption process acting on an otherwise clean signal, because then the question "is this defect present?" becomes "how would I detect this specific corruption?", which is a much more tractable engineering problem:

• Signal-to-noise ratio. Sensor noise, motion blur, compression artifacts, and dropped frames can corrupt observations. Without a model that separates observation noise from process noise, the learned transition distribution may attribute measurement variability to the environment. That misattribution can distort uncertainty estimates used by a planner; the direction of the control error depends on how the planner uses uncertainty, as discussed in Stochasticity and Uncertainty.

• Temporal alignment. The action paired with $(o_t,o_{t+1})$ must match the timing convention used by the model. Actuation latency, clock drift between the control loop and the camera, and buffered logging can break this correspondence. The data may still look plausible frame by frame while the fitted model learns a delayed or smeared action effect. Detecting the defect therefore requires checking action-observation timing rather than individual frames alone.

• Action fidelity. Commanded actions, measured joint torques, and realized end-effector velocities are different variables. Training with one action representation and querying the model with another creates a systematic interface mismatch. More data expressed through the same mismatched interface does not identify the missing mapping; calibration, paired measurements, or an explicit actuator model can.

• Episode completeness. Truncated episodes can create false terminal targets if the training pipeline treats the final recorded frame as a genuine terminal state, pads it as stationary, or otherwise ignores censoring. If truncation correlates with failures, an unmodeled termination indicator can also leak logging behavior into the learned dynamics.

• Label and state-estimate correctness. If states come from a filter or tracker, estimation failures can be absorbed into the learned dynamics unless uncertainty or invalidity is represented. A simulator can expose its internal state directly, avoiding a tracker in simulator coordinates, although those states and the simulated physics can still differ from reality.

The uncomfortable part is that quality is model-relative and task-relative. Repeated noisy observations can help a stochastic model identify a deployment-relevant noise process, but only when the noise is represented appropriately and can be distinguished from latent dynamics. Aggressive, jerky actions may be irrelevant to one task and essential for learning actuator saturation in another. Before you write a filter, name the downstream consumer, the decisions it will make, and which quality dimensions affect those decisions. Filtering removes information as well as corruption, so retain the raw, provenance-linked source whenever policy permits.

It is tempting to reduce diversity to a single number and optimize it. Don't. Two distinct quantities matter:

• Coverage asks: which regions of the reachable state-action space have support in the data? A query outside that support requires extrapolation, where accuracy is not constrained directly by observations.
• Density asks: how many samples fall in each region under a chosen representation and resolution? Density affects estimation error within a region, but sample count alone does not guarantee a good fit or calibrated uncertainty.

Coverage and density problems can produce different symptoms, but neither has a unique signature. Out-of-support queries may yield implausible rollouts or deceptively smooth ones, while sparse in-support regions may show high error without an explicit rise in predictive variance. Nearest-neighbor distances, histograms, and kernel-density estimates in a task-relevant embedding are diagnostics rather than proofs: their conclusions inherit the representation and distance metric.

The plot below puts the two quantities side by side on the raw dataset. Density is the number of points packed into a region; coverage is whether a region has any points at all. The narrow cloud saturates the nearby region, while the wide cloud reaches far from the origin but is thin.