Data Curation and Synthetic Data › A Worked Curation Pipeline
World Models HandbookAll chapters Data Curation and Synthetic Data › A Worked Curation PipelineOne section · Sections in this chapter6 / 9 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
Let us build the pipeline end to end on a small, fully controlled system where the data-generating dynamics are known. The experiment measures how each step changes one fixed linear estimator; it does not estimate how valuable that step will be for a different model or task. We use a two-dimensional nonlinear system with control input:
where:
- : the state at time (two-dimensional)
- : the control input at time
- : the state transition matrix
- : the control input matrix
- : a small scalar coefficient controlling the strength of the cubic nonlinearity
- : the elementwise cube of , i.e.,
- : process noise, a two-dimensional Gaussian with zero mean and covariance
- : the process noise standard deviation
- : the identity matrix
The cubic term makes the dynamics nonlinear, with a contribution that grows with state magnitude. A linear model fitted only in the small-state regime cannot represent that term and can extrapolate poorly at larger magnitudes. Wider-state data reveals the misspecification, although data alone cannot make the linear class express the cubic dynamics. This distinction becomes important when we interpret balancing below.
We deliberately construct a problematic dataset: it contains exact duplicates, is dominated by a narrow region of state space, and includes 40 observation-corrupted episodes out of 160. We then run deduplication, quality filtering, balancing, and synthetic augmentation, measuring autoregressive rollout error on held-out trajectories from both regions. Feeding each prediction back as the next model input exposes compounding transition error. It is still not a full control evaluation because no planner selects actions from the learned model.
Building the deliberately bad dataset
We start with the known dynamics and a trajectory generator. Actions follow a slowly varying AR(1) process rather than independent draws, so action jitter contributes less to the roughness score used later. Every episode has the same small process-noise scale; the corrupted group receives additional independent observation noise after latent-state generation. This distinction matters because filtering legitimate process stochasticity would change the environment regime rather than repair a sensor defect. The smooth action process is an experimental design choice, not a universal property of real control logs: when commanded actions contain abrupt changes, a quality detector should condition on them rather than equating state roughness with corruption.
import numpy as np
# Ground-truth nonlinear dynamics, with weak control authority.
A_TRUE = np.array([[0.93, 0.11], [-0.09, 0.86]])
B_TRUE = np.array([[0.06, 0.00], [0.00, 0.06]])
CUBIC = 0.05
HORIZON = 40def make_episodes(n, process_noise, rng, s0_scale=1.0, observation_noise=0.0):
"""Generate latent dynamics, then optionally corrupt the observations."""
episodes = []
for _ in range(n):
s0 = s0_scale * rng.uniform(-1.0, 1.0, size=2)
action = np.zeros(2)
actions = []
for _t in range(HORIZON):
action = 0.97 * action + 0.20 * rng.standard_normal(2)
actions.append(action.copy())
actions = np.asarray(actions)
state = s0.copy()
states = [state.copy()]
for a in actions:
state = A_TRUE @ state + B_TRUE @ a - CUBIC * state**3
state = state + process_noise * rng.standard_normal(2)
states.append(state.copy())
latent_states = np.asarray(states)
if observation_noise > 0:
observed_states = (
latent_states
+ observation_noise * rng.standard_normal(latent_states.shape)
)
else:
observed_states = latent_states.copy()
episodes.append(
{
"states": observed_states,
"actions": actions,
"s0": observed_states[0].copy(),
"process_noise": process_noise,
"observation_noise": observation_noise,
"scale": s0_scale,
}
)
return episodesNow we assemble the corrupt dataset. It has four populations: 80 clean narrow episodes, 40 noise-corrupted narrow episodes, only 10 wide-region episodes, and 30 exact copies of the first 30 clean episodes. The imbalance is intentional and exaggerated, so that every curation step has an obvious effect on the resulting model. In real data the imbalance is usually milder but is spread across many more sources, which is what makes it so hard to spot by eye.
rng = np.random.default_rng(20240517)
clean = make_episodes(80, process_noise=0.01, rng=rng, s0_scale=1.0)
for ep in clean:
ep["group"] = "clean-narrow"
noisy = make_episodes(
40,
process_noise=0.01,
rng=rng,
s0_scale=1.0,
observation_noise=0.35,
)
for ep in noisy:
ep["group"] = "noisy-narrow"
wide = make_episodes(10, process_noise=0.01, rng=rng, s0_scale=3.0)
for ep in wide:
ep["group"] = "clean-wide"
duplicates = [
{
"states": ep["states"].copy(),
"actions": ep["actions"].copy(),
"s0": ep["s0"].copy(),
"process_noise": ep["process_noise"],
"observation_noise": ep["observation_noise"],
"scale": ep["scale"],
"group": "duplicate",
}
for ep in clean[:30]
]
raw_dataset = clean + noisy + wide + duplicatesEpisodes in raw dataset : 160
Transitions per episode : 40
Total reported transitions: 6,400
clean-narrow: 80 episodes
noisy-narrow: 40 episodes
clean-wide: 10 episodes
duplicate: 30 episodesThe headline count is misleading. Thirty of the "episodes" are byte-identical copies, so about 19% of the reported dataset carries no new information at all. That is the first thing deduplication will fix.
Scoring quality
Our quality score is the mean Euclidean norm of the second difference of the observed state sequence. For an additive scalar observation-noise sequence with independent terms of variance , the noise component of has variance . The score therefore amplifies independent high-frequency noise, but it also responds to action changes and genuine nonlinear acceleration. It is model-free in the sense that it does not fit a dynamics model; it is not assumption-free.
def roughness(episode):
"""Mean Euclidean norm of the second difference of the state sequence."""
second_diff = np.diff(episode["states"], n=2, axis=0)
return float(np.mean(np.linalg.norm(second_diff, axis=1)))
for ep in raw_dataset:
ep["roughness"] = roughness(ep)
scores_by_group = {
g: np.array([ep["roughness"] for ep in raw_dataset if ep["group"] == g])
for g in ["clean-narrow", "clean-wide", "duplicate", "noisy-narrow"]
} clean-narrow: mean roughness = 0.0260 max = 0.0308
clean-wide: mean roughness = 0.0433 max = 0.0716
duplicate: mean roughness = 0.0266 max = 0.0297
noisy-narrow: mean roughness = 1.1212 max = 1.4169In this seeded toy dataset, the mean roughness of the noisy group is more than an order of magnitude above each clean-group mean. Let us inspect the full distribution before selecting a threshold.
The orange marks are medians. In this constructed example every high-noise episode has a larger roughness score than every clean episode, including the wide-region trajectories whose larger states produce more curvature. A quantile threshold need not lie in that gap, however: the 75th-percentile rule below retains seven of the 40 noisy episodes because it is defined to keep 75% of the deduplicated data. When score distributions overlap, conditional or multi-signal rules may preserve rare valid dynamics better than a global cutoff.
Deduplication and quality filtering
Now the mechanics. We deduplicate with an exact fingerprint over the initial state and the first few actions, then filter the noisiest quartile.
def signature(episode, decimals=8):
"""Cheap exact-match fingerprint: initial state plus the first few actions."""
key = np.concatenate([episode["s0"], episode["actions"][:4].ravel()])
return tuple(np.round(key, decimals))
def deduplicate(episodes):
seen, kept = set(), []
for ep in episodes:
sig = signature(ep)
if sig in seen:
continue
seen.add(sig)
kept.append(ep)
return kept
deduped = deduplicate(raw_dataset)
scores = np.array([ep["roughness"] for ep in deduped])
threshold = float(np.quantile(scores, 0.75))
filtered_deduped = [ep for ep in deduped if ep["roughness"] <= threshold]raw : 160 episodes {'clean-narrow': 80, 'clean-wide': 10, 'duplicate': 30, 'noisy-narrow': 40}
after dedup : 130 episodes {'clean-narrow': 80, 'clean-wide': 10, 'noisy-narrow': 40}
roughness threshold (q75): 0.9692
after quality filter : 97 episodes {'clean-narrow': 80, 'clean-wide': 10, 'noisy-narrow': 7}Deduplication removes the 30 constructed copies. The 75th-percentile quality filter removes 33 of the 40 noisy episodes and retains all 90 clean episodes, including the 10 wide-region episodes. Those counts validate the rule only for this seeded construction. In a real project, select the statistic and threshold on a development split, report per-regime retention, and test sensitivity to the cutoff.
Balancing coverage of the state space
What remains is clean but radically imbalanced. The wide-region episodes are a tiny fraction of the whole, so any model fitted to this data will be dominated by the narrow regime. We rebalance by resampling the smaller bucket with replacement up to the size of the larger one.
def region_label(episode):
return "wide" if episode["scale"] > 2.0 else "narrow"
def resample_balanced(episodes, regions=("narrow", "wide"), seed=0):
"""Resample every region up to the size of the largest region."""
local_rng = np.random.default_rng(seed)
buckets = {
r: [ep for ep in episodes if region_label(ep) == r] for r in regions
}
non_empty = {r: b for r, b in buckets.items() if b}
target = max(len(b) for b in non_empty.values())
balanced = []
for region, bucket in non_empty.items():
idx = local_rng.integers(0, len(bucket), size=target)
balanced.extend(bucket[i] for i in idx)
return balanced, target
counts_before = {
r: sum(region_label(ep) == r for ep in filtered_deduped)
for r in ("narrow", "wide")
}
balanced_deduped, target_per_region = resample_balanced(
filtered_deduped, seed=3
)
counts_after = {
r: sum(region_label(ep) == r for ep in balanced_deduped)
for r in ("narrow", "wide")
}before balancing: {'narrow': 87, 'wide': 10} (total 97)
after balancing: {'narrow': 87, 'wide': 87} (total 174)
target per region: 87
Balancing is not free. Oversampling the ten wide episodes up to parity repeats each of them many times, deliberately changing their loss contribution without adding support. Generating new wide-region samples can avoid exact repetition, but only by importing assumptions from the generator. The next step demonstrates that trade-off.
Synthetic augmentation
We fit a linear transition model to the cleaned toy data and use it as a lightweight generator, sampling fresh wide-region initial states and the same action process used in the real-data generator. This is not independent evidence: the synthetic transitions inherit the fitted linear model's bias. They increase the number of distinct sampled trajectories in a target region, while their fidelity remains bounded by the learned generator.
def fit_linear_model(episodes):
"""Least-squares fit of s_{t+1} = A s_t + B a_t over all transitions."""
x = np.concatenate(
[np.hstack([ep["states"][:-1], ep["actions"]]) for ep in episodes]
)
y = np.concatenate([ep["states"][1:] for ep in episodes])
theta, *_ = np.linalg.lstsq(x, y, rcond=None)
return theta
def step(theta, state, action):
"""Apply a fitted linear transition model."""
return state @ theta[:2] + action @ theta[2:]
theta_clean = fit_linear_model(clean)True A: [[ 0.93 0.11] [-0.09 0.86]] Fitted A from clean narrow episodes only: [[ 0.913 0.108] [-0.09 0.847]]
For this seeded fit, the diagonal entries are smaller than those of . The cubic term contributes state-dependent contraction, and least squares absorbs part of its average effect into the fitted linear coefficients. At larger state magnitudes, the omitted cubic contribution is stronger, so the linear model cannot represent the true transition with one constant matrix. Additional narrow-regime data can reduce estimation variance but cannot identify large-state behavior that is absent from the sample or remove this model-class misspecification.
def synthesize(theta, n, s0_scale, process_noise, rng, label="synthetic"):
"""Roll out a fitted model to produce new, region-targeted episodes."""
episodes = []
for _ in range(n):
s0 = s0_scale * rng.uniform(-1.0, 1.0, size=2)
action = np.zeros(2)
actions = []
for _t in range(HORIZON):
action = 0.97 * action + 0.20 * rng.standard_normal(2)
actions.append(action.copy())
actions = np.asarray(actions)
state = s0.copy()
states = [state.copy()]
for a in actions:
state = step(theta, state, a) + process_noise * rng.standard_normal(
2
)
states.append(state.copy())
episodes.append(
{
"states": np.asarray(states),
"actions": actions,
"s0": s0,
"process_noise": process_noise,
"observation_noise": 0.0,
"scale": s0_scale,
"group": label,
"roughness": roughness({"states": np.asarray(states)}),
}
)
return episodes
theta_gen = fit_linear_model(filtered_deduped)
synthetic_wide = synthesize(
theta_gen,
n=40,
s0_scale=3.0,
process_noise=0.01,
rng=rng,
label="synthetic-wide",
)The generator inherits the linear model's misspecification and the weighting of its fitting set. It starts trajectories in the wide region, but those transitions follow the learned linear dynamics rather than the known cubic dynamics. The synthetic episodes therefore increase nominal regional coverage while remaining imperfect in fidelity. We evaluate that distinction against real held-out trajectories from each region below.
Comparing data mixtures
We now fit the same linear model class to six versions of the same source and evaluate each by autoregressive rollout on held-out trajectories from both regions. The model's predictions are fed back as inputs while the recorded actions remain fixed. This measures open-loop action-sequence prediction with closed-loop state recursion; it does not measure a planner choosing actions from the model. We also include a balanced, clean "reference collection" of 150 narrow and 150 wide episodes. It is a data reference, not an oracle model: the fitted predictor is still linear while the true dynamics are cubic.
# Held-out evaluation trajectories from both regions.
eval_rng = np.random.default_rng(99)
eval_narrow = make_episodes(15, process_noise=0.01, rng=eval_rng, s0_scale=1.0)
eval_wide = make_episodes(15, process_noise=0.01, rng=eval_rng, s0_scale=3.0)
eval_sets = {"narrow": eval_narrow, "wide": eval_wide}
# A clean, balanced reference collection for the same misspecified model class.
reference = make_episodes(
150, process_noise=0.01, rng=rng, s0_scale=1.0
) + make_episodes(150, process_noise=0.01, rng=rng, s0_scale=3.0)def rollout_error(theta, episodes):
"""Mean per-step error when the model feeds its own predictions back in."""
errors = []
for ep in episodes:
state = ep["states"][0].copy()
for action, target in zip(ep["actions"], ep["states"][1:]):
state = step(theta, state, action)
errors.append(float(np.linalg.norm(state - target)))
return float(np.mean(errors))
mixtures = {
"raw": raw_dataset,
"dedup": deduped,
"dedup + filter": filtered_deduped,
"dedup + filter + balance": balanced_deduped,
"dedup + filter + balance + synth": balanced_deduped + synthetic_wide,
"reference mix": reference,
}
mixture_errors = {}
for name, data in mixtures.items():
theta = fit_linear_model(data)
mixture_errors[name] = {
k: rollout_error(theta, v) for k, v in eval_sets.items()
} mixture | episodes | narrow | wide
----------------------------------------------------------------------
raw | 160 | 0.172 | 0.284
dedup | 130 | 0.178 | 0.299
dedup + filter | 97 | 0.110 | 0.163
dedup + filter + balance | 174 | 0.119 | 0.175
dedup + filter + balance + synth | 214 | 0.116 | 0.172
reference mix | 300 | 0.102 | 0.145
Read the chart with three questions in mind:
- What did deduplication and filtering change? Exact deduplication alone moves mean per-step error from 0.172 to 0.178 in the narrow region and from 0.284 to 0.299 in the wide region. Equal copies change the empirical objective but do not guarantee a better estimator. Filtering then lowers the errors to 0.110 and 0.163, respectively, in this seeded run.
- Did balancing solve the coverage problem? No. Resampling to parity raises the errors to 0.119 and 0.175. The wider states expose stronger cubic dynamics, but the fitted class is linear. Repeating those episodes cannot express the missing term and shifts the compromise fit in a way that hurts both reported regions. This is why balancing must be validated with the intended model class rather than judged by counts.
- What did synthetic augmentation contribute? Relative to the resampled mixture, adding learned synthetic wide trajectories lowers the errors to 0.116 and 0.172, but it does not beat the filtered-only mixture. The synthetic generator shares the linear model's bias. The clean balanced reference collection performs best in this run (0.102 narrow, 0.145 wide), while retaining error from the misspecified linear model. Better collection policies could improve the evidence available to a suitable model; we return to that idea in Active Perception, Dual Control, and Exploration.
The synthetic-augmented mixture is not best on wide-region error in this run. Here we know the generator is biased because it omits the cubic term; in a real pipeline the result could also reflect sampling variance, optimization, or an unsuitable target mixture. Synthetic data can expand nominal coverage, but fidelity must be tested on real held-out trajectories.