Probing Classifiers › Probing Task Design

The value of a probing experiment lives or dies by the quality of its task design. The task must be test linguistic information, have a clear definition, and matched to the kind of representation being probed.

Probing tasks roughly divide into three categories based on the level of linguistic abstraction they target. Syntactic tasks ask about grammatical structure, the kind of information that traditional NLP systems modeled explicitly with parsing algorithms. Semantic tasks ask about meaning-level properties that depend on context and world knowledge. Surface tasks ask about superficial statistical features that a model could plausibly encode without deep understanding. Comparing probe performance across these three levels tells you something important about the nature of the representations: models that encode only surface features are very different from models that capture linguistic structure.

The most studied probing tasks involve syntax. Part-of-speech tagging asks whether a token's representation encodes its grammatical category (noun, verb, adjective, etc.). Syntactic dependency labels ask whether the relation between a head and its dependent (subject, object, modifier) is encoded. Both tasks have gold-standard annotated corpora (Penn Treebank, Universal Dependencies) that provide large amounts of labeled data.

Constituent tree depth probing asks a more subtle question: given the representation of a token, can a probe predict how deep that token sits in the syntactic parse tree? A noun embedded inside a prepositional phrase inside a relative clause sits deeper than a main-clause subject. The finding that BERT representations encode tree depth (Jawahar et al., 2019) suggests the model builds something like an internal parse tree even when trained only on raw text with masked-language modeling.

Grammatical number agreement is another well-studied syntactic probe. Given a sentence like "The keys on the table are on the shelf," does the representation of "are" encode that "keys" (not "table") is the subject that determines plural agreement? Probes for subject-verb number agreement test whether long-distance syntactic dependencies are represented, not just local bigram-level patterns. The surprising finding is that BERT representations support this kind of long-range agreement even across intervening noun phrases, suggesting that something functionally analogous to syntactic structure guides how representations are formed.

Syntactic dependency probing goes further by asking about the labeled arc structure of a parse. For a sentence parsed as a dependency tree, each word has a head (the word it depends on) and a dependency label (the grammatical function of the relation, such as subject, object, or modifier). Probing whether the representation of a word predicts its dependency label, independent of its POS tag, tests whether syntactic function is encoded separately from syntactic category. The answer, across multiple studies, is that it is: dependency labels are accessible, and they carry information beyond what POS alone would predict.

Semantic probing tasks ask about meaning-level properties. Named entity type prediction is common: does the representation of "Paris" distinguish it from "Microsoft" as a location versus an organization? Semantic role labeling probes ask whether the representation of a word in a sentence encodes its thematic role as agent, patient, or instrument.

Coreference probes examine whether the model represents that two mentions in a text refer to the same entity. A pair of representations for "John" and "he" in the same paragraph should, if coreference is encoded, be more similar when "he" refers to "John" than when it refers to a different individual. Word sense disambiguation probes ask whether the representation of a polysemous word like "bank" reflects which sense is active in context. These tasks are harder to probe successfully because they depend on long-range context and compositional semantics.

Semantic similarity probing tests whether the cosine distance between two word representations correlates with human judgments of semantic relatedness. This is a regression task rather than a classification task, and it measures something slightly different: not whether a discrete property is encoded, but whether the geometry of the representation space reflects semantic relationships.

Sentiment-relevant feature probing has also been studied: given a movie review, does the representation of a word encode whether it is more likely in positive or negative reviews? This connects probing to interpretability for NLP applications rather than just linguistic science.

There is a real risk in probing research of probing surface statistical regularities rather than linguistic structure. If your training corpus is dominated by financial news, models might learn that "bank" in financial contexts tends to appear near "interest rate" and "deposits." A probe might detect "financial institution" sense not because the model understands word sense disambiguation, but because it has memorized co-occurrence statistics.

This motivates careful task design that controls for surface confounds. Good probing tasks:

• Have balanced class distributions across varied contexts
• Are tested on out-of-domain examples to check for generalization
• Separate linguistic structure from corpus-level statistics
• Are defined precisely enough that two annotators would agree on the correct label

The position-in-sentence probe is a canonical example of a surface task. A probe trained to predict whether a token appears in the first or second half of the sentence can succeed simply because BERT's positional embeddings encode position directly. This has nothing to do with deep linguistic understanding. Including such tasks alongside linguistic tasks lets you calibrate what "high accuracy" means: if the surface task achieves 99% accuracy and the syntactic task achieves 88%, the 88% looks more impressive. If the surface task achieves only 65%, the 88% is even more striking.

Token length is another surface task. Short function words (the, a, it, in) have different syntactic roles on average than long content words (subsequently, particularly, generalization). A probe for token length might accidentally proxy for syntactic category without capturing grammatical structure.

Another design choice involves granularity. POS tagging at the Universal POS level has 17 coarse categories. Penn Treebank POS has 45 fine-grained categories. Syntactic function labels number in the dozens. The coarser the task, the higher the probe accuracy, but the less informative the result. Knowing that a model can linearly separate nouns from verbs is less interesting than knowing it can distinguish 17 syntactic roles in complex embedded clauses.

Researchers often run probes at multiple granularities to understand how easily different levels of linguistic structure can be extracted. The pattern typically shows that coarse distinctions are encoded linearly and accessibly, while fine-grained distinctions require either deeper layers or more expressive probes.

This granularity tradeoff has a theoretical interpretation. Coarse POS categories (noun, verb, adjective) are highly predictive of surface distributional patterns: nouns appear after determiners, before verbs, and in positions where noun phrases occur. Models trained on raw text with any prediction objective will learn to distinguish these categories because their surface distributions are so different. Fine-grained categories (proper noun vs. common noun, main verb vs. auxiliary) are less predictable from local distributional statistics and require more sophisticated contextual analysis. The difficulty of probing fine-grained categories thus tells you something about the depth of the representations.