State, Physics, Causality, and Memory Evaluation › State Probing and Representation Quality

A probe is a small model fitted on top of a frozen representation. You take the encoder, stop its gradients, run it over a dataset, and then ask: can a simple readout predict quantity $q$ from the representation $z_t$? Saying that the representation "contains" $q$ can conflate recoverability by a chosen readout, unrestricted information content, and downstream use. We need to ask how the information is arranged and which procedure can read it out.

The three questions are:

1. Linear accessibility. How well does a fitted linear readout recover $q_t$ from $z_t$ on held-out data? Success demonstrates accessibility to that fitted readout on the tested distribution. Failure can reflect regularization, sample size, fitting, or distribution shift; it does not prove that no linear readout could succeed. A linear probe still learns a task-specific mapping.
2. Information decodability. Can some function in a richer probe family recover $q_t$? A flexible probe can learn substantial task structure from its frozen inputs, so high performance depends jointly on the encoded information, the readout's capacity, and its fitting procedure. Capacity cannot supply target information absent from those inputs.
3. Downstream functional use. Does the rest of the system use $q_t$ to make decisions? This is the only one of the three that connects to behaviour, and it cannot be established by a probe at all. A probe is a diagnostic instrument attached to the side of the model; it does not tell you what the model does with the information when it acts.

Comparing readouts can reveal which fitted family works better on the test distribution. Better nonlinear scores do not establish impossibility of linear recovery, and failures under both families do not prove absent information. Good decoding also does not establish causal necessity: the downstream system may ignore the recovered variable. Testing functional use requires additional experiments, such as controlled representation interventions and closed-loop comparisons.

Check these three procedural details before interpreting a probe score. Splits and preprocessing determine what information is held out; controls clarify how the selected probe behaves on an alternative task. Mistakes can bias an evaluation or limit the conclusions it supports.

The probe itself is a fitted map $g_\theta$ from the frozen representation $z_t$ to a target $q_t$. It is a simple readout, not a full model. In the linear case, $g_\theta(z_t) = W z_t + b$; in the nonlinear case, the probe first maps $z_t$ through a fixed feature function and then applies a ridge readout. The data are the held-out trajectories, and all probe scores are computed against targets that were not used to fit either the encoder or the probe, and trajectories are held out as complete sequences rather than individual frames so that frames from one rollout never appear in two splits. The key design choice is that the encoder is frozen: the probe cannot reshape the representation to make its job easier, so any information the probe recovers must already be present in $z_t$.

Split at the trajectory level, not the frame level. Consecutive frames of one trajectory are strongly correlated: neighbouring timesteps share nearly the same position. Random frame splits can put near-duplicates of test frames in the training set and inflate the probe's held-out score through memorisation. Splitting whole trajectories removes this within-trajectory leak. In our toy we hold out complete rollouts. Choose a split unit that respects the dependencies in the data; shared scenes or subjects may require a larger grouping than one trajectory.

Fit preprocessing on training data only. Standardisation statistics, whitening matrices, and PCA bases must be estimated on the training split and then applied unchanged to validation and test. Fitting them on the full dataset uses held-out information. Its effect on a score depends on the data and fitting procedure; neither inflation nor a flat train/test gap is guaranteed.

Use a control task with matched probe capacity. Hewitt and Liang (2019) developed type-level randomized control tasks for linguistic probes. Our shuffled regression targets are a simpler toy control, not a reproduction of their protocol. Report real-target and control results together, with the randomization and split specified. A control gives context about the chosen fitting procedure; it does not bound all structured-task learning or certify information content.

To keep this concrete, let us build a frozen representation by hand and be explicit about what it is. We will use a phase encoding of position only. In the formula and NumPy code, $x_t$ and $y_t$ denote the numerical values of observed coordinates in metres: the dimensionless phase is $2\pi$ times the coordinate divided by a one-metre period.

where:

• $x_t$: the observed horizontal position at time $t$, in metres.
• $y_t$: the observed vertical position at time $t$, in metres.
• $\cos(2\pi x_t), \sin(2\pi x_t)$: the phase encoding of the horizontal coordinate as a point on the unit circle; together they determine its numerical value modulo one. Exact inversion over a full period is nonlinear; this does not preclude a useful approximate linear readout over a restricted distribution.
• $\cos(2\pi y_t), \sin(2\pi y_t)$: the phase encoding of the vertical coordinate as a point on the unit circle, with the same periodic-recovery property.

This is a handcrafted encoder, not a learned one. Exact recovery of a noiseless position within a specified one-period interval uses nonlinear phase inversion; positions differing by an integer share the same encoding, and noisy observations near the interval boundary complicate recovery. The encoder does not directly take velocity, but that does not make velocity statistically independent of its output. Our sampled trajectories can correlate position and velocity. The experiment tests these readouts on a named distribution, not an absolute present-versus-absent information theorem.

Our world is a point mass in the unit square with perfectly reflecting walls. Let us fix the conventions before writing any metric. Fixing conventions up front is not a formality; it is what makes the numbers in the tables below interpretable and comparable across sections.

• State: $s_t = (x_t, y_t, v^x_t, v^y_t)$, metres and metres per second.
• Action: $a_t$, metres per second squared, applied as an acceleration.
• Time step: $\Delta t = 0.1$ s, so one trajectory of 30 steps is 3 seconds.
• Observation: $o_t = (x_t, y_t) + \epsilon_t$ with independent coordinate noise $\epsilon_t \sim \mathcal{N}(0, (0.004\ \text{m})^2 I_2)$.
• Occluder: the band $x \in [0.5 - w, 0.5 + w]$ for $y \in [0.05, 0.95]$, with $w$ the occluder half-width.
• Data-generating policy: $a_t \sim \mathrm{Uniform}(-0.3, 0.3)^2$ m/s², independent across components and timesteps.
• Split unit: one complete trajectory. We use 60 for training, 20 for validation, 20 for test.

The hidden fraction tells us how much of the data sits behind the occluder. At 0.278 it is not small, and occlusion is not rare in this dataset. The reason occlusion matters out of proportion to its frequency is that a rare failure to maintain a coherent hidden state can dominate the downstream consequences, even if the model is otherwise accurate on 90% of the data.

We now build the probe dataset. Because the encoder is defined on observed positions, we keep only timesteps where the ball is visible, and we standardise the representation using training statistics only. Standardising with statistics computed on the full dataset is the leakage trap described above, so we are careful to compute the mean and standard deviation from the training split and apply the same transformation to validation and test.

Interpret the probe scores with three limits in mind: readout capacity, failed recovery, and downstream use.

A flexible probe may be doing substantial task learning from the information available in its frozen inputs, not creating target information missing from them. A nonlinear readout's gain can depend on both the encoded features and the readout's capacity and fitting procedure; the score does not apportion these contributions. Our control records this procedure's held-out performance after target shuffling. It bounds neither all learning on random targets nor learning on structured targets. Report the probe architecture, feature dimension, and regularisation strength so that the procedure can be reproduced.

Low linear performance does not show that information is absent. Failure of both chosen families still cannot rule out successful recovery by another readout or training procedure. A control task provides context, not an information-theoretic absence test. What you have shown is how these specific probe procedures performed on this split.

Good decoding does not establish causal necessity. If a fixed readout recovered velocity exactly almost surely throughout the target population, then $v_t=g(z_t)$ on that population. Zero error on a finite test set does not establish that condition. Neither result shows that the transition model or policy uses the recovered velocity. Controlled representation interventions can test that use, but an intervention inside a model is not automatically an intervention in the world. Chapter 66 returns to this distinction in interpretability and world-model debugging.

There is a further trap in our toy: the encoder is handcrafted, so this experiment characterises a diagnostic procedure, not a trained world model. Its inputs are known by construction, but statistical relationships between position and velocity still depend on the trajectory distribution. A real encoder requires its own splits, controls and interpretation.