Catastrophic Forgetting › A Worked Example: Tracing Forgetting Step by Step

To make the mechanics concrete, let us trace through a small numerical example of how catastrophic forgetting unfolds. Imagine a toy neural network with just three parameters, $\theta_1$, $\theta_2$, and $\theta_3$, trained to perform two tasks sequentially. This example is deliberately simple so every step is transparent, but the same dynamics operate in networks with billions of parameters.

Setup. Suppose our network has the following pre-trained weights after Task A training:

$$\boldsymbol{\theta}^* = [\theta_1^*, \theta_2^*, \theta_3^*] = [1.0, 0.5, -0.3]$$

Task A performance depends heavily on $\theta_1$ and $\theta_2$ (high importance), while $\theta_3$ has low importance for Task A. After pre-training, the model achieves 88% accuracy on Task A.

Fine-tuning on Task B. Now we fine-tune on Task B. The gradient computed from the first batch of Task B data is:

$$\nabla_{\boldsymbol{\theta}} \mathcal{L}_B = [0.4, -0.3, 0.6]$$

This gradient says: to reduce Task B loss, increase $\theta_1$ by 0.4, decrease $\theta_2$ by 0.3, and increase $\theta_3$ by 0.6. With a learning rate of $\alpha = 0.1$, the update rule $\theta \leftarrow \theta - \alpha \nabla \mathcal{L}_B$ produces:

$$\boldsymbol{\theta}^{(1)} = [1.0 - 0.04, \ 0.5 + 0.03, \ -0.3 - 0.06] = [0.96, \ 0.53, \ -0.36]$$

After one step: $\theta_1$ drifted from 1.0 to 0.96, $\theta_2$ from 0.5 to 0.53. These changes are small, but notice that $\theta_1$, which is highly important for Task A, moved in the direction needed for Task B, not in the direction needed for Task A. The gradient for Task B is not aware of what Task A needs.

Accumulation over many steps. After 100 gradient steps with similar Task B gradients, $\theta_1$ might have drifted to 0.6 and $\theta_2$ to 0.8. Task A's performance, which depended on $\theta_1 \approx 1.0$ and $\theta_2 \approx 0.5$, drops dramatically: perhaps from 88% to 60%, a loss of nearly 30 percentage points in absolute terms.

If we applied EWC regularization, the Fisher information values for these parameters would be approximately $F_1 = 0.9$, $F_2 = 0.7$, $F_3 = 0.1$ (high importance for $\theta_1$ and $\theta_2$, low for $\theta_3$). The regularized gradient for $\theta_1$ would be:

$$\nabla_{\theta_1} \mathcal{L}_{\text{EWC}} = 0.4 + \lambda \cdot F_1 \cdot (\theta_1 - \theta_1^*) = 0.4 + 0.5 \cdot 0.9 \cdot (1.0 - 1.0) = 0.4$$

In the first step the EWC gradient is the same, because $\theta_1$ has not yet drifted. But after drift begins, the EWC regularization term grows to resist further movement:

$$\nabla_{\theta_1} \mathcal{L}_{\text{EWC}} = 0.4 + 0.5 \cdot 0.9 \cdot (0.96 - 1.0) = 0.4 - 0.018 = 0.382$$

The regularization pushes $\theta_1$ back toward 1.0 while the task gradient pushes it down. The larger the drift, the larger the push-back. After 100 steps, $\theta_1$ might stabilize near 0.85 rather than drifting to 0.6, preserving more of Task A's performance. This is how regularization-based mitigation works: it reshapes the loss surface so that the optimizer encounters resistance when it tries to move important parameters too far from their pre-trained values.