Transfer Learning › Probing What Models Learn
Language AI HandbookAll chapters Transfer Learning › Probing What Models LearnOne section · Sections in this chapter9 / 13 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
Probing tasks are simple classification tasks designed to test whether specific linguistic properties are encoded in model representations. By training a lightweight classifier on top of frozen model representations, you can assess what information is accessible in different layers.
Probing gives a window into the internal representations of pre-trained models. The methodology is elegant: extract representations from a specific layer, freeze those representations, and train a simple logistic regression or MLP classifier to predict a linguistic property (such as part-of-speech tag, syntactic dependency label, or semantic role). If the simple classifier reaches high accuracy, the information is present and accessible in that layer's representations. If it fails, the information is either absent or encoded in a way the classifier cannot access.
Probing complements the qualitative layer analysis we did earlier. Where the visualization showed that word senses separate across layers, probing gives us quantitative evidence for specific linguistic properties. Together, these analyses paint a detailed picture of how linguistic knowledge is organized within pre-trained transformer models.
Let's implement a simple probing experiment to verify that syntactic information is encoded in pre-trained representations:
# Probing for part-of-speech information in BERT representations
from transformers import AutoModel, AutoTokenizer
# Prepare probing data: words with their POS tags
# 18 nouns, 18 verbs, 18 adjectives = 54 examples total for stable CV
probing_data = [
# Nouns
("The cat sat on the mat.", [("cat", "NOUN"), ("mat", "NOUN")]),
("She read a book yesterday.", [("book", "NOUN")]),
("The dog chased the ball.", [("dog", "NOUN"), ("ball", "NOUN")]),
(
"The teacher explained the concept.",
[("teacher", "NOUN"), ("concept", "NOUN")],
),
(
"A storm destroyed the bridge overnight.",
[("storm", "NOUN"), ("bridge", "NOUN")],
),
(
"The artist finished her painting.",
[("artist", "NOUN"), ("painting", "NOUN")],
),
(
"The child found a coin on the street.",
[("child", "NOUN"), ("coin", "NOUN"), ("street", "NOUN")],
),
(
"Her garden was full of flowers.",
[("garden", "NOUN"), ("flowers", "NOUN")],
),
(
"The pilot landed the plane safely.",
[("pilot", "NOUN"), ("plane", "NOUN")],
),
# Verbs
("He runs every morning.", [("runs", "VERB")]),
("They decided to leave early.", [("decided", "VERB"), ("leave", "VERB")]),
("The bird sings beautifully.", [("sings", "VERB")]),
("She writes letters to her friends.", [("writes", "VERB")]),
("The engineer designed a new system.", [("designed", "VERB")]),
("The children played in the park.", [("played", "VERB")]),
("He forgot to lock the door.", [("forgot", "VERB"), ("lock", "VERB")]),
("The scientists discovered a new compound.", [("discovered", "VERB")]),
("She teaches mathematics at the university.", [("teaches", "VERB")]),
# Adjectives
("The red car is fast.", [("red", "ADJ"), ("fast", "ADJ")]),
("She wore a beautiful dress.", [("beautiful", "ADJ")]),
("The old house was empty.", [("old", "ADJ"), ("empty", "ADJ")]),
(
"The narrow path was slippery after rain.",
[("narrow", "ADJ"), ("slippery", "ADJ")],
),
(
"He gave a brief but clear explanation.",
[("brief", "ADJ"), ("clear", "ADJ")],
),
(
"The fragile vase sat on a wooden shelf.",
[("fragile", "ADJ"), ("wooden", "ADJ")],
),
(
"She was tired and unhappy after the meeting.",
[("tired", "ADJ"), ("unhappy", "ADJ")],
),
("The crispy bread tasted fresh.", [("crispy", "ADJ"), ("fresh", "ADJ")]),
("It was a dark and stormy night.", [("dark", "ADJ"), ("stormy", "ADJ")]),
]
tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")
model = AutoModel.from_pretrained(
"bert-base-uncased", output_hidden_states=True
)
model.eval()def extract_word_representations(sentence, target_word, layer=6):
"""Extract representation for a specific word from a specific layer."""
inputs = tokenizer(sentence, return_tensors="pt")
tokens = tokenizer.tokenize(sentence)
# Find target word position
target_lower = target_word.lower()
word_idx = None
for idx, token in enumerate(tokens):
if token.startswith(target_lower) or target_lower.startswith(
token.replace("##", "")
):
word_idx = idx + 1 # +1 for [CLS]
break
if word_idx is None:
return None
with torch.no_grad():
outputs = model(**inputs)
hidden_state = outputs.hidden_states[layer]
return hidden_state[0, word_idx, :].numpy()
# Build probing dataset
X, y = [], []
pos_to_idx = {"NOUN": 0, "VERB": 1, "ADJ": 2}
for sentence, word_tags in probing_data:
for word, pos in word_tags:
rep = extract_word_representations(sentence, word, layer=6)
if rep is not None:
X.append(rep)
y.append(pos_to_idx[pos])
X = np.array(X)
y = np.array(y)## Train a simple probe and evaluate
from sklearn.model_selection import cross_val_score
probe = LogisticRegression(max_iter=1000, random_state=42)
scores = cross_val_score(
probe, X, y, cv=min(3, len(X) // 3), scoring="accuracy"
)POS Probing Accuracy: 0.98 (+/- 0.03) Number of probing examples: 46 Representation dimensionality: 768
Even with this tiny probing dataset, the classifier reaches reasonable accuracy at distinguishing parts of speech. This shows that BERT's layer 6 representations encode syntactic information. The success of this simple experiment shows the rich linguistic knowledge that BERT acquired during pre-training. Large-scale probing studies use thousands of examples and show that different layers specialize in different linguistic properties, with lower layers encoding morphology, middle layers encoding syntax, and upper layers encoding semantics.
To see how syntactic information is distributed across layers, we can extend our probing experiment to test each layer:
# Probe POS information across all layers
layer_probing_accuracies = []
for layer in range(13):
X_layer, y_layer = [], []
for sentence, word_tags in probing_data:
for word, pos in word_tags:
rep = extract_word_representations(sentence, word, layer=layer)
if rep is not None:
X_layer.append(rep)
y_layer.append(pos_to_idx[pos])
if len(X_layer) >= 6:
X_layer = np.array(X_layer)
y_layer = np.array(y_layer)
probe_layer = LogisticRegression(max_iter=1000, random_state=42)
n_splits = min(3, len(X_layer) // max(len(set(y_layer)), 1))
if n_splits >= 2:
layer_scores = cross_val_score(
probe_layer, X_layer, y_layer, cv=n_splits
)
layer_probing_accuracies.append(layer_scores.mean())
else:
probe_layer.fit(X_layer, y_layer)
layer_probing_accuracies.append(probe_layer.score(X_layer, y_layer))
else:
layer_probing_accuracies.append(np.nan)import matplotlib.pyplot as plt
use_book_style()
plt.rcParams["figure.figsize"] = (6.0, 4.0)
plt.figure()
plt.plot(range(13), layer_probing_accuracies, "o-", linewidth=1, markersize=8)
plt.xlabel("Layer")
plt.ylabel("POS Probing Accuracy")
plt.title("Syntactic Information Across BERT Layers")
plt.xticks(range(13), ["Emb"] + [str(i) for i in range(1, 13)])
plt.ylim(0, 1.05)
plt.show()
This layer-wise analysis confirms the hierarchical organization of linguistic knowledge in pre-trained models. Syntactic information like part-of-speech tags becomes increasingly accessible as we move from the embedding layer through the early transformer layers, typically reaching peak accessibility in middle layers. The slight decline in upper layers shows their specialization toward more abstract semantic representations that may not require explicit syntactic encoding.
The probing methodology also has broader implications for transfer learning practice. If you know which layer best encodes the type of information relevant to your task, you can make informed decisions about how many layers to fine-tune, which layers to freeze, and how to architect your task-specific head. For tasks primarily requiring morphological information, lower layers matter most. For tasks requiring syntactic parsing, middle layers are most informative. For tasks requiring semantic understanding, upper layers are the necessary resource.
Key Parameters
The key parameters for the probing experiment are:
- output_hidden_states (
AutoModel): Enables access to internal layer representations required for probing. - cv (
cross_val_score): The number of folds for cross-validation. This keeps reliable performance estimation. - max_iter (
LogisticRegression): Ensures the probe classifier converges given the high-dimensional input features.