Probing Layers › Probing Variants and Extensions

The basic layer probing setup, where you train a linear probe at each layer and plot accuracy, has spawned a rich family of methodological extensions that address its limitations or ask more refined questions.

John Hewitt and Christopher Manning introduced structural probing (2019) as a more geometrically sophisticated alternative to linear classification probes. Instead of asking whether a linear classifier can predict discrete labels (like POS tags), structural probing asks whether a linear transformation of the representation space can recover the entire syntactic tree structure.

The structural probe learns a matrix $\mathbf{B}$ such that the squared distance between transformed representations of two tokens $i$ and $j$ approximates the tree distance between them in the ground-truth parse:

$$d_T(i, j) \approx \left\| \mathbf{B} \mathbf{h}_i - \mathbf{B} \mathbf{h}_j \right\|^2$$

where:

• $d_T(i, j)$: the number of edges in the shortest path between tokens $i$ and $j$ in the dependency parse tree
• $\mathbf{B}$: a learned linear transformation (projection matrix)
• $\mathbf{h}_i, \mathbf{h}_j$: the representations of tokens $i$ and $j$

This is a more demanding test than classification probing. It asks not just "can a probe distinguish noun from verb?" but "does the geometry of the representation space organize tokens in a way that mirrors the topology of their syntactic relationships?" If the answer is yes, then syntactic structure is not just implicitly encoded; it is geometrically structured in the representation.

Hewitt and Manning found that BERT's representations at middle layers pass this structural test with surprisingly high fidelity. The representation space at layers 5-7 can be linearly transformed to approximately recover parse tree distances, suggesting that BERT's representations have internalized tree structure as a geometric property. This is a much stronger claim than what linear classification probing can make: syntactic relationships between tokens are encoded as distances in the representation space, which is a geometric analog of the symbolic parse tree.

The structural probe also introduced a different notion of what "probing" means. Rather than training a classifier on fixed representations, you are fitting a geometric transformation to the representation space. The probe's success tells you about the intrinsic geometry of the representations, not just their predictive content.

A significant challenge with standard probing is deciding when a probe is "too complex." A probe with more parameters can always achieve higher accuracy by memorizing, which inflates the measure of how much information is encoded. Pimentel et al. (2020) proposed an information-theoretic framework that addresses this directly.

In this framework, you measure the mutual information $I(P; R)$ between the probe target $P$ (e.g., POS labels) and the representation $R$ (e.g., the hidden states). Mutual information captures how much knowing the representation reduces uncertainty about the target, regardless of how that information is geometrically organized. The advantage is that mutual information is probe-agnostic: it measures the information content of the representation, not the capacity of the probe to access it.

In practice, estimating mutual information from finite samples requires approximations, often done by using the negative cross-entropy of a trained classifier as a lower bound on the mutual information. This brings the approach back to training classifiers, but with a different interpretive frame: the classifier's performance is now a lower bound on the mutual information rather than a direct measure of linear accessibility.

The information-theoretic view clarifies the relationship between probe complexity and interpretation. A linear probe with high accuracy on a non-linearly encoded property would have a high negative cross-entropy (suggesting high mutual information) even though the property is not linearly accessible. A probe that fails because the property is absent would have low mutual information regardless of probe complexity. This framework provides cleaner separation between "the property is not encoded here" and "the property is encoded non-linearly."

Voita and Titov (2020) introduced minimum description length (MDL) probing as another way to address the probe complexity problem. The key insight is that a good representation should allow a probe to classify correctly with a concise model. If you need to transmit a probe model along with the representation to achieve correct predictions, the cost of that transmission (the description length) is a measure of how non-obvious the encoding is.

MDL probing frames the task as a compression game. You want to find the probe that, together with the representation, minimizes the total description length of the probe and the labeled data. A representation that encodes a property linearly requires a very simple probe (short description length). A representation where the property is deeply buried requires a complex probe (long description length). The MDL framework automatically penalizes complex probes, which prevents inflation from memorization.

In practice, MDL probing is implemented by measuring how many bits are needed to transmit the probe (using online coding schemes) and comparing this to the bits needed to describe the data without the probe (using a uniform prior). The ratio gives a measure of how much the representation compresses the task. This technique has produced more reliable layer localization results than accuracy-based probing on small datasets, where overfitting is a concern.