ELECTRA › The Replaced Token Detection Objective

The RTD objective trains the discriminator to predict is_replaced for every token. Let's formalize the setup before presenting the loss function.

Problem setup:

• We start with an original sequence $\mathbf{x} = [x_1, x_2, \ldots, x_n]$ containing $n$ tokens
• A random subset of positions $\mathcal{M} \subset \{1, \ldots, n\}$ is selected for masking (typically 15% of positions)
• The generator receives the sequence with [MASK] tokens at positions in $\mathcal{M}$ and produces replacement tokens $\tilde{x}_i$ for each $i \in \mathcal{M}$
• The corrupted sequence $\tilde{\mathbf{x}}$ combines original and generated tokens: $\tilde{x}_i = x_i$ for positions not in $\mathcal{M}$, and $\tilde{x}_i$ equals the generator's sampled token for positions in $\mathcal{M}$

Note that even at masked positions, the generator might sample the same token as the original. These positions are technically "original" for the discriminator's task, since no replacement occurred.

The discriminator outputs logits $D(\tilde{\mathbf{x}})_i$ for each position. The RTD loss is the standard binary cross-entropy, summed across all positions in the sequence:

$$\mathcal{L}_{\text{RTD}} = -\sum_{i=1}^{n} \left[ y_i \log \sigma(D_i) + (1 - y_i) \log (1 - \sigma(D_i)) \right]$$

where:

• $n$: the total number of tokens in the sequence (e.g., 512)
• $D_i = D(\tilde{\mathbf{x}})_i$: the discriminator's raw logit output at position $i$, representing how confident the model is that the token was replaced
• $\sigma(\cdot)$: the sigmoid function, which converts the logit to a probability between 0 and 1
• $y_i = \mathbb{1}[\tilde{x}_i \neq x_i]$: the ground truth label, equal to 1 if the token at position $i$ was replaced by the generator, and 0 if it's the original token

The loss decomposes into two cases: when $y_i = 1$ (token was replaced), the loss is $-\log \sigma(D_i)$, encouraging the model to output a high logit. When $y_i = 0$ (token is original), the loss is $-\log(1 - \sigma(D_i))$, encouraging a low logit. The sum runs over all $n$ positions, including the unmasked ones. This is the main difference from MLM: every token contributes to learning.