Security, Ethics, Privacy, and Governance › Privacy in Recorded Environments

This section covers the information a world model absorbs about the people in its training data and what happens when that information can be recovered.

A recorded-environment corpus may contain continuous, context-rich personal information. Curated object datasets can also include identifying people or spaces; this is a difference to assess in the actual corpus, not a clean dividing line between model families. Possible examples include:

• Home recordings that show private interiors.
• Driving recordings that retain recognizable pedestrians or license plates.
• Clinical recordings that retain patient bodies or movements.
• Social recordings that retain conversations, faces, or contextual cues useful for identification.

Separate information present in the corpus from information retained or exposed by the model. Synthetic, task-specific, or privacy-processed datasets may differ. Continuous motion and contextual relationships can warrant additional privacy assessment where they concern identifiable people.

Consider three possible exposure channels. Their applicability depends on whether the corpus contains personal data, what the model returns, and what the attacker can observe.

• Dataset leakage. The corpus itself escapes: a breach, an unsecured bucket, a scraped copy. This exposes stored recordings directly rather than demonstrating leakage through model outputs. The amount and sensitivity of exposed personal information can increase potential harm. Assess the consequences for affected people in the incident's context.
• Model-level leakage. Membership inference tests whether a candidate record was in the training set (Shokri et al., 2017; Carlini et al., 2022). Attribute inference targets hidden properties. Certain diffusion-model attacks have recovered near-identical training images, including identifiable people (Carlini et al., 2023, "Extracting Training Data from Diffusion Models"; Somepalli et al., 2023). That is evidence of leakage in those studied systems, not proof that every generative world model regenerates personal records. Realistic or confident outputs alone do not establish memorization.
• Interaction leakage. Observable actions or response timing can supply candidate signals for a membership attack. Faster navigation in a familiar hallway, for example, could also reflect geometry, hardware or ordinary generalization. Establish leakage with a defined candidate population, controlled alternatives and measured false-positive rates; familiarity alone is not a membership oracle.

Differential privacy supplies a formal distributional guarantee under a specified adjacency relation. A randomized mechanism $M$ is $(\varepsilon, \delta)$-differentially private if, for any two neighboring datasets $D$ and $D'$, and for every measurable output set $S$,

$$\Pr[M(D) \in S] \le e^{\varepsilon} \, \Pr[M(D') \in S] + \delta$$

where:

• $\varepsilon$: the privacy budget; with $\delta$ and the adjacency relation fixed, smaller values tighten the multiplicative bound on how one neighboring-data change affects output probabilities
• $\delta$: an additive slack term in the inequality, not generally the probability of a single identifiable failure event
• $M$: the randomized mechanism; the probabilities range over its internal randomness
• $D,D'$: neighboring datasets under the chosen record, trajectory, or person-level adjacency
• $S$: any measurable set of possible outputs
• The bound controls the change in output probabilities between neighbors; its interpretation depends on $\varepsilon$, $\delta$, and the adjacency definition

Differentially private stochastic gradient descent (Abadi et al., 2016) is the workhorse for training.

Three structural features of world models make DP harder to apply than the textbook case suggests.

First, one person can contribute many records. Record-level adjacency and person-level adjacency are different protection units. For pure record-level $(\varepsilon,0)$-DP, a group of $k$ records has the bound $(k\varepsilon,0)$ (Dwork and Roth, Theorem 2.2). With approximate DP, the additive term does not in general merely scale as $k\delta$: iterating the inequality gives $\delta\sum_{i=0}^{k-1}e^{i\varepsilon}$, capped at one. Group privacy is distinct from composition of repeated mechanisms. A person-level guarantee can instead be designed with person-level contribution bounds and accounting; temporal correlation does not invalidate the definition itself.

For the elementary pure-DP group bound only, holding group epsilon at one requires record epsilon at most $1/k$. The plot illustrates that bound, not an implemented DP training mechanism or a measured privacy–utility frontier.

Second, trajectory patterns can identify people. In the mobility dataset studied by de Montjoye et al. (2013), four randomly chosen spatio-temporal points uniquely identified 95% of individuals. This is a result about that dataset and resolution, not every trajectory corpus. Removing direct identifiers can reduce exposure while leaving re-identification through patterns possible. Recorded human trajectories warrant a different assessment from synthetic or nonpersonal transition data.

Third, task utility and identifying detail can overlap. Some tasks require geometry or dynamics that also reveal a private environment. This motivates measuring a privacy–utility tradeoff for the actual task and protection unit. It does not prove that useful prediction always requires identifiable recordings: task-specific sensing, aggregation, synthetic data, and privacy-preserving training can change the tradeoff.

Federated learning can keep raw examples at participating devices. Secure aggregation is designed to reveal an aggregate rather than each participant's individual contribution under its stated assumptions. Deep Leakage from Gradients demonstrates reconstruction when an attacker observes the gradients needed by that attack; it is not a demonstration that a properly configured secure aggregate reveals the same information. Evaluate aggregate sizes, participation, auxiliary information and repeated access separately. Temporal structure may be relevant to an attack, but this chapter supplies no world-model reconstruction experiment or universal claim that it makes reconstruction easier.

Let's evaluate a loss-threshold membership attack without assuming that it succeeds. We reuse the synthetic nonlinear plant and fit 200 smooth Fourier features plus an intercept to 150 transitions. Feature count alone does not imply 201 independent directions or exact interpolation: the feature matrix is numerically rank-deficient on these correlated trajectories. We use separate nonmember trajectories for threshold calibration and evaluation, so the reported false-positive rate is measured out of the calibration sample. No personal records are used.

These outputs do not show exact memorization or a powerful membership oracle. Smooth, dependent features and ridge regularization limit interpolation, despite the nominal feature count. The attack rates vary with regularization but do not decrease monotonically to chance. The meaningful comparison is TPR versus measured held-out FPR; the nominal calibration percentile is not a universal no-information baseline. A weak result from one seeded attack does not establish privacy against stronger attacks.

The defensible lesson is methodological: specify attacker knowledge, calibrate independently, report the operating point and finite-sample uncertainty, and keep privacy claims separate from prediction accuracy. Overfitting can create membership signals in some settings, but tighter fit does not universally imply more leakage. This experiment is not differentially private: ridge regularization, a loss histogram, and seeded randomness do not supply a DP guarantee. Formal DP instead requires a randomized mechanism, a stated adjacency relation, and valid privacy accounting, as developed by Dwork and Roth (2014).

• Collect less. Avoiding an unnecessary sensitive recording removes that direct corpus exposure; it does not rule out information inferred through other channels. Consider task-specific sensors, raw-data deletion where legally and technically feasible, and bounded retention. Derived features and model outputs need their own assessment.
• Separate the shared model from local adaptation. A design that keeps personal adaptation local can reduce central data transfer. Verify whether parameters, outputs, telemetry, backups or updates leave the device, and include local compromise in the threat model. Local training alone does not imply that deployment-specific information never leaves.
• Evaluate regularization, and measure leakage. Report attack TPR alongside measured FPR, model utility, calibration data, and candidate-record assumptions. A failed attack is not proof of anonymity.
• Differential privacy with contribution accounting. Choose a record, session, household, or person as the adjacency unit, and bound all contributions by that unit. A session-level bound does not protect a person who contributes multiple sessions without additional accounting. Report the mechanism, accountant, parameters, and utility; no universal utility penalty is established here.
• Lawful processing and bystanders. Recorded people may not be the operator. Identify the applicable jurisdiction, lawful basis, notice, retention, and rights processes; consent is not the only lawful basis in every legal system. Minimize collection independently of the operator's willingness to record. Ethical stakeholder consultation is separate from a legal-compliance claim.