Replaced Token Detection › Why Detection Is Easier Than Generation
Language AI HandbookAll chapters Replaced Token Detection › Why Detection Is Easier Than GenerationOne section · Sections in this chapter5 / 17 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
The efficiency gain comes from the difference between detection and generation. Predicting the exact token that was masked requires learning fine-grained distinctions in a vocabulary of 30,000+ tokens. Detecting whether a token was replaced requires only a binary decision at each position.
Consider a masked position in the sentence "The [MASK] barked loudly." To predict the correct token, the model must assign probability to "dog" over thousands of alternatives. But to detect a replacement, the model only needs to recognize that "cat" or "piano" in that position feels wrong, even if it cannot pinpoint exactly what should be there.
This asymmetry allows the discriminator to learn useful representations from every position. Original tokens that fit the context well should score high. Replaced tokens, even plausible ones, often have subtle mismatches with surrounding context. The model learns to detect these mismatches, developing representations that capture semantic coherence.
The key insight here is about the nature of the learning signal. In MLM, the only positions that generate a gradient are the masked ones. Every other position is frozen during the backward pass. In RTD, the discriminator must decide "original or replaced?" at every position, and this decision requires the model to build a contextual understanding of whether each token fits its surroundings. That understanding is precisely what makes a strong encoder: the ability to represent each token in relation to everything around it.
Think of the discriminator as constantly asking the question, "Does this word belong in this sentence, given everything else around it?" Even when the answer is always "yes, it's original," answering that question confidently requires the model to understand how words relate to each other. This is why the discriminator, despite being trained on a seemingly simpler task, develops representations that transfer well to complex downstream tasks like question answering and natural language inference.
There is also a subtle but important point about what makes RTD's replacements challenging. The generator is not inserting random words. It is sampling from a language model that has learned which words are likely in a given context. This means the replacements tend to be grammatically plausible and semantically related. The discriminator must distinguish "researcher" from "scientist" in "The [word] discovered a cure," a much harder detection task than identifying "refrigerator" in the same position. This difficulty level is self-adjusting: as the generator improves, its replacements become more plausible, forcing the discriminator to encode finer contextual distinctions.
def detection_vs_generation_example():
"""Illustrate the difference between detection and generation tasks."""
sentence = "The chef cooked the meal"
masked_position = 1 # "chef" is masked
# Generation: must predict exact token from 30k vocabulary
generation_task = {
"input": "The [MASK] cooked the meal",
"target": "chef",
"vocabulary_size": 30522,
"task": "Predict exactly which token was masked",
}
# Detection: binary classification at each position
detection_task = {
"input": "The waiter cooked the meal",
"labels": ["original", "replaced", "original", "original", "original"],
"task": "Is each token original or replaced?",
}
return generation_task, detection_task
gen_task, det_task = detection_vs_generation_example()Generation Task (MLM): Input: The [MASK] cooked the meal Must predict: 'chef' from 30,522 options Detection Task (RTD): Input: The waiter cooked the meal Labels: ['original', 'replaced', 'original', 'original', 'original'] Task: Binary classification at each position
The generator deliberately produces challenging replacements. If it simply inserted random tokens, detection would be trivial. By sampling from a language model, the generator creates replacements that are semantically plausible but contextually imperfect. This forces the discriminator to distinguish finer contextual differences.