Continual Learning Problem › Continual Learning Problem

Every model you have studied so far in this book learned from a fixed dataset. You gathered data, split it into training and validation sets, trained until the model converged, and deployed. When the world changed or new data arrived, the standard response was to retrain from scratch. This works reasonably well when data fits comfortably in memory and training is cheap. But the real world rarely cooperates on either count.

Imagine a language model deployed to assist customer support agents. In January, users ask about product features. By March, a new product launches and brings new terminology. By June, a competitor event shifts how customers describe their problems. If you retrain the model from scratch each time, you spend enormous compute, require large amounts of old data to remain on hand, and introduce deployment lag. If you continue training on new data alone, something alarming happens: the model forgets what it learned before.

This forgetting is not a minor nuisance. It is a fundamental and well-characterized failure mode called catastrophic forgetting. Understanding it, formalizing it, and eventually mitigating it is the subject of the continual learning research area. This chapter establishes the problem: what continual learning means, why catastrophic forgetting occurs at the level of gradient descent and parameter space, how different learning scenarios shape the difficulty of the problem, and what the specific failure modes look like in practice. Subsequent chapters in this part of the book will develop the tools to address these problems.