Continual Learning Evaluation › Limitations and Practical Considerations

The metrics described in this chapter capture different facets of the memory-plasticity tradeoff, but they carry important limitations that practitioners should understand before relying on them to guide research decisions or deployment choices.

Task boundary dependence. BWT, FWT, and the forgetting measure all require clear task boundaries: you need to know when training on one task ends and the next begins, so you can record the diagonal entries of the accuracy matrix. In real-world continual learning, task boundaries are often implicit or unknown. A deployed model receiving a steady stream of data does not know when the distribution has shifted. The shift from one user behavior pattern to another, from one product category to another, or from one linguistic register to another, may be gradual and unlabeled. Evaluation protocols that require explicit boundaries do not directly translate to these scenarios. Researchers have begun developing boundary-free evaluation protocols that measure forgetting through held-out test sets sampled at regular time intervals, but these are less established and not yet standardized across the literature.

The benchmark gap. Permuted MNIST, Split CIFAR, and similar benchmarks have well-known failure modes: they are small, they involve simple classification tasks, and they use fixed task sequences that may not reflect realistic deployment patterns. A method that achieves state-of-the-art BWT on permuted MNIST may fail completely on real-world data streams where tasks are noisy, overlapping, or presented in non-stationary order. Tasks in practice are rarely clean binary or 5-class problems with sharp boundaries. They are more often continuous distributions that shift gradually, with overlapping class spaces and varying data quality. Evaluation on multiple benchmarks with different characteristics, including at least one that tests distributional shift (like CORe50) and one that involves natural language, is necessary to build confidence in a method's real-world performance.

Task identity leakage. Many evaluation protocols claim to test class-IL or domain-IL settings, but implicitly allow the model to infer task identity from context. For instance, if each task uses a different subset of output classes, a model that observes which output logits are active during training can infer the current task at test time. If a model has been trained to classify only classes 0 and 1 during task 1, it may learn to produce near-zero logits for all other classes. At test time, the model can effectively determine it is being queried about task 1 by observing which logits are large. Careful benchmark design and evaluation protocols are needed to prevent this leakage, including scrambling output indices across tasks and using uniform random initialization of output heads at each task transition.

Metric gaming. Each summary metric can be optimized at the expense of others, and researchers who are not careful may inadvertently design methods that improve a reported metric without improving actual continual learning capability. A method could achieve near-zero BWT by using a separate model for each task (progressive neural networks), but this would not scale to many tasks and would have trivial FWT. Average accuracy can be maximized by a method that simply allocates more capacity to the last task seen, at the expense of all earlier tasks, if the benchmark's final evaluation happens to weight recent tasks more. When comparing methods, it is important to report the full set of metrics rather than cherry-picking the one that favors a particular approach. The community has gradually moved toward requiring that papers report at least average accuracy, BWT, and FWT together, though the adoption of forgetting and intransigence as standard reports remains incomplete.

The replay data assumption. Methods that store and replay past examples require that past data remains accessible indefinitely, or at least for the duration of the training sequence. In many practical scenarios, data from earlier tasks is deleted for privacy, storage, or regulatory reasons. A medical AI system trained on patient data from hospital A cannot legally retain that data when it moves to hospital B. Evaluation that includes replay-based methods alongside replay-free methods must make clear what data access assumptions each method requires, otherwise the comparison is unfair. This is especially important because replay methods consistently achieve the best benchmark numbers, and a practitioner who cannot legally retain past data would be misled by a leaderboard that does not distinguish replay-constrained from unconstrained methods.

Sensitivity to task order. The accuracy matrix, and all metrics derived from it, depend on the order in which tasks are presented. A task sequence where the most difficult task comes first will produce different BWT values than a sequence where the same task comes last, because the model has more or fewer subsequent tasks to overwrite its weights. Most benchmarks use a fixed task order, which means results are specific to that order and may not generalize to other orderings. Some papers evaluate methods across multiple random orderings and report mean and variance of metrics. This is a more thorough and honest evaluation approach, but it multiplies the computational cost by the number of orderings evaluated and is not yet universal practice.

The continual learning versus multi-task learning comparison. Average accuracy in continual learning is most naturally compared to the joint model trained on all tasks simultaneously. But the joint model is not always the right reference point. In scenarios where data arrives truly sequentially and storing all past data is impossible, the joint model may not be a feasible baseline. In those cases, the more relevant comparison might be to a model that stores only a small buffer of past examples (a constrained replay baseline) rather than the entire training set. Framing the comparison carefully, and stating explicitly what baselines are achievable in the target deployment scenario, makes evaluation results much more actionable.