Probing Layers › Probing Layers

When a transformer processes the sentence "The bank by the river flooded," how does it figure out that "bank" means a riverbank and not a financial institution? The model must resolve this ambiguity somewhere inside its stack of layers, and identifying where that resolution happens is exactly what layer probing sets out to answer.

As we saw in the previous chapter on Probing Classifiers, a probe is a simple supervised model trained on top of frozen representations. The key insight is that if a linear probe can predict a linguistic property from a set of representations, those representations encode that property. Layer probing extends this idea across all layers simultaneously: instead of probing a single fixed layer, you extract representations from every layer in the model and train a separate probe at each one. The resulting accuracy profile across layers tells you which layers are responsible for which types of linguistic or semantic information.

Think of it like excavating an archaeological site by taking soil samples at different depths. The artifacts you find at each stratum tell you about the civilization that existed at that level. Layer probing takes the same approach with a neural network: you sample the model at each computational depth and examine what kinds of linguistic knowledge are deposited there. Just as earlier strata hold older, more primitive materials while deeper strata hold refined artifacts from more complex societies, earlier transformer layers tend to hold coarser linguistic features while later layers accumulate richer semantic meaning.

This technique has become one of the most widely used tools in interpretability research. It lets you ask targeted questions like: "At which layer does BERT start distinguishing parts of speech?" or "Does syntactic structure emerge before semantic meaning, or do they appear together?" The answers have shaped how researchers think about the internal organization of transformer models and have practical implications for tasks like transfer learning, model compression, and parameter-efficient fine-tuning. Understanding where information lives inside a model is the first step toward understanding how the model makes decisions, and ultimately toward building more transparent, reliable systems.

The central appeal of layer probing lies in its simplicity: you do not need to modify the model, understand its internal computations, or make assumptions about how it works. You just ask a question about each layer's output and observe whether the answer is accessible. This non-invasive quality makes probing studies widely reproducible and applicable to any architecture that produces layer-by-layer hidden states.