Probing Classifiers › What Is a Probing Classifier?

The mechanics are straightforward. You take a pretrained model and freeze its weights entirely. You then pass a dataset of labeled examples through the model and collect the hidden-state representations at some intermediate layer. Finally, you train a small classifier, typically logistic regression, on those frozen representations to predict the labels.

The classifier is intentionally simple. A linear model (logistic regression) is the standard choice because it can only succeed if the information is linearly decodable from the representations. If the probe must itself solve the task through complex nonlinear computation, you can no longer attribute the success to the base model's representations. Using a small multilayer perceptron (MLP) as a probe is sometimes done, but it introduces ambiguity: did the probe find the information in the representations, or did it compute it from scratch?

The workflow is:

1. Choose a linguistic task with labeled data (e.g., part-of-speech tagging, syntactic dependency labeling, named entity type prediction).
2. Pass each example through the pretrained model, extracting the representation at a specific layer for a specific token.
3. Train a linear classifier on those representations with the linguistic labels as targets.
4. Evaluate on a held-out test set.
5. Interpret the accuracy as evidence of what the representations encode.

This workflow looks like standard supervised learning because it is. The key departure from normal supervised learning is that the features (the representations) come from a frozen pretrained model, not from the task-specific training process. Everything the representations can tell you was determined before the probe was trained.

The term "encoding" deserves scrutiny. When researchers say that BERT "encodes" POS information, what exactly do they mean?

The formal definition traces back to information theory. A representation $\mathbf{h}$ encodes a property $y$ if the mutual information $I(\mathbf{h}; y)$ is greater than zero:

where:

• $H(y)$ is the entropy of the label distribution, measuring how uncertain the label is without any representation
• $H(y \mid \mathbf{h})$ is the conditional entropy of the label given the representation, measuring residual uncertainty after seeing the representation
• $I(\mathbf{h}; y)$ is the reduction in uncertainty: how much knowing the representation tells you about the label

If mutual information is zero, the representation tells you nothing about the label. The property is completely absent. If mutual information equals $H(y)$, the representation determines the label perfectly.

The problem is that computing mutual information exactly for high-dimensional continuous representations is intractable. Probing accuracy is used as a proxy. A probe that achieves perfect accuracy corresponds to zero conditional entropy, which corresponds to perfect mutual information. In practice, probe accuracy gives you a lower bound on mutual information, not an exact estimate. Pimentel et al. (2020) analyze this relationship formally and show that the connection between probe accuracy and information content is subtle: different probe architectures and training procedures can give different accuracy estimates for the same underlying mutual information.

The choice of probe architecture matters more than it might seem. A linear probe is interpretable in a strong sense: it succeeds only if the linguistic property is accessible as a linear function of the representation space. This maps onto a meaningful geometric intuition. If part-of-speech classes are linearly separable in the representation space, a hyperplane exists that correctly partitions noun vectors from verb vectors. You can visualize this, compute feature importances, and reason about what dimensions of the representation the probe exploits.

Nonlinear probes, such as shallow MLPs with ReLU activations, relax this requirement. They can succeed even when the information is encoded in a nonlinear way. But this creates an interpretability problem: the probe's success might reflect its own computational power, not the structure of the representations. The worry is that a sufficiently powerful probe would succeed on random representations, defeating the purpose of the experiment.

This concern has led to a general norm in the field: use the simplest probe that can answer your question. If a linear probe achieves 95% accuracy on POS tagging, there is no reason to use an MLP. If a linear probe achieves only 60% and an MLP achieves 85%, the gap tells you that the information exists but is not linearly organized, which is itself informative.

The geometric intuition is worth making explicit. Consider a simplified 2D representation space. If a linear probe can separate POS classes, each class occupies a roughly convex region of the space and a straight line (or hyperplane in higher dimensions) suffices to partition them. The probe's weight vector defines the orientation of this hyperplane. If classes overlap in a way that only a curved boundary can separate them, a linear probe fails but an MLP might succeed. The following visualization demonstrates both scenarios using synthetic 2D data, illustrating why the choice between linear and nonlinear probes encodes a meaningful claim about the structure of the representation space.