Data Curation and Synthetic Data › Data Curation and Synthetic Data

Consider a log with 160 episodes and 6,400 transitions. Thirty episodes are exact copies, 40 have severe observation noise, and only 10 begin in the wide-state regime where the dynamics are most nonlinear. The transition count is correct, yet it is a poor description of what the dataset can teach. Later in this chapter, we will build exactly this controlled example and measure what deduplication, filtering, balancing, and synthetic augmentation change. The result is more useful than a slogan that curation always helps: filtering improves this particular misspecified model, while aggressive balancing can hurt because data cannot repair the model class. Curation determines both the evidence presented to the learner and the trade-offs hidden by an aggregate count.

That is why data curation deserves its own chapter rather than a paragraph in a training-loop appendix. A world model is fitted to statistics of its training distribution. If that distribution is dominated by a robot sitting still, an ordinary average-risk objective emphasizes stillness over uncommon motion. If noisy sensor logs dominate, a model without an explicit observation-noise mechanism can conflate measurement error with dynamics. If procedurally generated trajectories from one physics engine dominate, simulator-specific regularities can be learned alongside the intended dynamics. These outcomes depend on the objective, architecture, and labels, but the common point is simple: changing which regions appear, and how they are weighted, changes the fitted model even when the underlying physical system is identical.

In Interaction Data and Passive Observation, we asked where transitions come from: teleoperation, autonomous play, human video, logged fleet data, scripted policies, and simulators. This chapter takes that raw stream as given and asks what to do with it. We will work through four problems that show up in every serious pipeline:

• Trajectory quality and diversity. What makes an episode worth training on, and how do we measure coverage of the state-action space without pretending coverage is the same thing as quality?
• Deduplication, filtering, and balancing. How to detect and remove redundancy, how to remove corruption without creating blind spots, and how to re-weight a skewed distribution toward the one your planner will query.
• Simulation and procedural generation. How synthetic data expands coverage, what it costs in fidelity, and why models trained on their own outputs quietly collapse.
• Data mixtures, provenance, and contamination. How to choose mixture weights, how to track where every episode came from, and why the usual random train/test split is a trap for world models specifically.

A theme runs through all four: curation is a modeling decision expressed through data. Every filter encodes an assumption about what the model should learn. Every mixture weight encodes a guess about where the policy will operate. Making those choices explicitly, and measuring their consequences, is the difference between data engineering and data science. The distinction is not merely aesthetic. A data engineer asks "is this pipeline correct?" A data scientist asks "what did this pipeline make my model believe, and is that belief true?" The rest of this chapter is about turning the second question into something you can systematically answer.