Catastrophic Forgetting › Measuring Forgetting

To manage forgetting, we first need to quantify it. Without measurement, forgetting remains invisible until deployment reveals unexpected failures. Several metrics capture different aspects of how models deteriorate after fine-tuning, and choosing the right metric depends on what aspects of performance matter most for your application. Each metric answers a slightly different question, so understanding their distinctions helps you select the right diagnostic for your situation.

The challenge with measuring forgetting is that you must evaluate the model on something other than what it was trained on. If you evaluate only on the fine-tuning task, you will see improvement or plateau, never forgetting. You need a held-out set of probes that test capabilities the model was not explicitly trained to preserve. These probes might be held-out test sets from prior tasks, standardized benchmarks, or simply general text perplexity measurements. The important thing is that they are orthogonal to the fine-tuning objective.

Measuring forgetting also forces a discipline that improves fine-tuning practice more generally. When you track prior capabilities systematically, you quickly discover that many standard fine-tuning choices (high learning rates, long training runs, no regularization) cause far more forgetting than practitioners realize. The measurement itself changes how you approach fine-tuning, making you more deliberate about learning rate selection, training duration, and regularization strength.

The most direct measure is backward transfer: how much does performance on prior tasks change after learning new ones? This metric captures the intuition that we want to compare the model's performance on old tasks before and after it learns something new. If the model performed well on named entity recognition before fine-tuning on sentiment analysis, backward transfer tells us how much of that NER performance survived.

To formalize this intuition, we need notation for tracking performance across multiple tasks learned sequentially. Imagine a model that learns task 1, then task 2, then task 3, and so on up to task $T$. At each point in this sequence, we can evaluate the model on any of the tasks it has encountered. The backward transfer metric averages the performance changes across all previously learned tasks, giving a single number that summarizes the net effect of subsequent training on prior capabilities.

The formula computes this average change as:

$$\text{BT} = \frac{1}{T-1} \sum_{i=1}^{T-1} (R_{T,i} - R_{i,i})$$

where:

• $\text{BT}$: the backward transfer score, which summarizes how much prior task performance has changed across the entire learning sequence
• $T$: the total number of tasks learned sequentially, representing the full learning trajectory
• $\frac{1}{T-1}$: the normalization term that computes the average change across all prior tasks, so the metric is comparable regardless of how many tasks were learned
• $R_{T,i}$: the performance on task $i$ after the model has finished training on all $T$ tasks, representing the model's current capability on that task
• $R_{i,i}$: the baseline performance on task $i$ immediately after it was originally learned, representing the best performance the model achieved when that task was its primary focus
• $\sum_{i=1}^{T-1}$: the summation over all previous tasks (excluding the current final task $T$), accumulating the performance changes across the model's learning history

Why does this formula make sense? Notice that each term $R_{T,i} - R_{i,i}$ is positive when the model performs better on task $i$ now than it did when it first learned that task (forward transfer helped) and negative when the model performs worse (forgetting occurred). By averaging these differences across all prior tasks, we get a single summary statistic. A backward transfer score of $-0.15$ means the model has lost an average of 15 percentage points of performance across previously learned tasks. A score near zero means prior capabilities were largely preserved. Positive backward transfer, while rare, indicates that learning new tasks somehow improved performance on old ones, often because the tasks share underlying structure.

A related metric tracks the maximum performance achieved on each task versus current performance. This metric addresses a subtle limitation of backward transfer: what if performance on a task fluctuates during training? Perhaps the model's NER performance increased slightly after learning sentiment analysis, before crashing when a third task was learned. Backward transfer would capture only the final change from initial to current, missing the fact that performance briefly improved and then deteriorated. The forgetting measure captures the worst-case degradation from peak performance throughout the entire training history.

The forgetting measure for task $i$ computes the gap between the highest performance ever recorded on that task and where performance stands at the end of training:

$$F_i = \max_{t \in \{1, \ldots, T-1\}} R_{t,i} - R_{T,i}$$

where:

• $F_i$: the forgetting measure for a specific task $i$, quantifying how much performance on that particular task has degraded from its peak
• $\max_{t} R_{t,i}$: the highest performance recorded for task $i$ at any previous time step $t$, representing the best the model ever did on this task throughout its training history
• $R_{t,i}$: the performance on task $i$ after training on task $t$, letting us to track performance at each point in the training sequence
• $R_{T,i}$: the current performance on task $i$ after training on the final task $T$, representing where the model stands now

Why does this formula make sense? Notice that the forgetting measure is always non-negative: the maximum over history is at least as large as the current value. A task might achieve peak performance not immediately after being learned, but at some later point when a related task was trained and introduced shared representations. The forgetting measure accounts for this by always comparing against the best-ever performance. This provides a more complete picture of capability loss. When $F_i$ is large, the model's current capability on task $i$ is far below its peak, and you should investigate what training events caused the degradation.

For language models, perplexity on held-out general text provides a task-agnostic forgetting measure. Perplexity quantifies how surprised the model is by text it has not seen before. Lower perplexity indicates better language modeling capability, as the model assigns higher probability to the actual words that appear. It is a natural diagnostic because language models were pre-trained specifically to predict text, and perplexity directly measures that capability.

If a model's perplexity on Wikipedia increases from 15 to 45 after fine-tuning on legal documents, that quantifies how much general language modeling capability was lost. The model has become three times more surprised by general English text, showing substantial degradation in its ability to model the distribution of ordinary language. This metric is particularly valuable because it does not require defining specific tasks. It measures the model's overall competence with language rather than its performance on any particular downstream application. Perplexity degradation is often the first warning sign practitioners notice when fine-tuning has gone too far.

The point is that perplexity on diverse general text is a proxy for the breadth of a model's language knowledge. A model that still assigns reasonable probabilities to varied sentences across domains and topics, written in different styles, has preserved its general language understanding. A model whose perplexity has spiked has become domain-locked, unable to process language outside the distribution it was fine-tuned on.

In practice, we use standardized benchmark suites to measure forgetting across tasks. These suites provide a battery of diverse tasks that together assess the breadth of a model's capabilities. For encoder models like BERT, the GLUE benchmark provides multiple tasks spanning sentiment analysis, textual entailment, and semantic similarity. Performance across this suite reveals whether fine-tuning has preserved the model's general language understanding or degraded it. The advantage of standardized benchmarks is comparability: you can measure your fine-tuned model against the published baselines and see exactly where it stands relative to the original pre-trained model.

For generative models, evaluations might include:

• Language modeling perplexity on diverse corpora
• Zero-shot performance on reasoning benchmarks
• Few-shot classification accuracy
• Generation quality metrics (fluency, coherence)

These complete evaluations provide a multidimensional view of forgetting, revealing which capabilities survived fine-tuning and which did not. A model might retain strong semantic understanding but lose syntactic precision. It might preserve factual recall but lose coherent long-form generation. Benchmark suites expose these asymmetries in a way that a single aggregate metric cannot.