Probing Classifiers › What Probing Research Has Found

Setting aside methodological subtleties, probing research has produced a consistent picture of what BERT-style models encode.

Syntactic structure is robustly encoded in middle layers. Tenney et al. (2019) probed for ten different linguistic tasks across BERT layers and found a clear stratification: basic features like POS and constituent labels peak in early-middle layers, while semantic tasks like coreference and semantic role labeling peak in later layers. The finding that the structure of language is layered inside BERT in roughly the same order as the NLP pipeline was striking enough to prompt the paper title "BERT Rediscovers the Classical NLP Pipeline." This result was not obvious: there was no reason in principle why the layers of a model trained on masked language modeling should correspond to the layers of the NLP processing hierarchy. The finding suggests that the hierarchical structure of language reflects how information needs to be processed to predict masked tokens effectively, rather than being a linguist.s organizational choice.

Syntactic trees are encoded in a geometric sense. Hewitt and Manning (2019) showed that the squared distance between words' representations in a transformed linear space correlates with their distance in the syntactic parse tree. This is a stronger claim than ordinary probing: syntactic information is accessible and encoded as a metric geometry in representation space. The structural probe they introduced finds a low-dimensional linear subspace of BERT's representation space (rank 64 out of 768 dimensions) in which tree distances are recovered with high accuracy. This means parse tree structure is concentrated in a structured subspace rather than distributed across all 768 dimensions, which has implications for how we think about where syntactic knowledge is "stored."

Semantic roles are partially encoded, but with more noise and ambiguity than syntax. The difficulty of semantic role labeling probes relative to POS probes reflects a property of these representations: semantics is less deterministic and more context-dependent than syntax. A verb like "run" takes different semantic arguments in "She runs a company" and "She runs a marathon," and the probe must somehow distinguish these uses. The partial success of semantic probes suggests that context-sensitive semantic information is present but harder to extract with a linear classifier.

World knowledge is surprisingly accessible via probing. Petroni et al. (2019) showed that filling in factual blanks ("The Eiffel Tower is in [MASK]") accesses information distributed across the transformer's representations. This is different from linguistic structure probing but uses the same methodology, and it demonstrates that the scope of what "encoding" means for large models extends beyond grammar to encyclopedic knowledge. The LAMA benchmark (LAnguage Model Analysis) quantified this: BERT's representations encode enough world knowledge to answer many factual questions in a cloze format, even though it was never trained on explicit question answering.

Number and gender agreement probes have also produced interesting findings. For languages with grammatical gender (French, German, Spanish), probes for noun gender consistently find that gender is encoded linearly at early-to-middle layers. This is less surprising than syntactic tree encoding, because gender in these languages is largely lexically determined and a model can learn gender from the distributional patterns of determiners and adjectives that agree with each noun. More interesting is the finding that agreement features are propagated through long dependency chains: the representation of a participle in a sentence with a complex noun phrase encodes the gender of the distant head noun, rather than the nearest noun.