Continual Learning Problem › What Is Continual Learning?

Continual learning, also called lifelong learning or sequential learning, refers to the setting where a model must learn from a non-stationary stream of tasks or data distributions over time. The model encounters each task or data batch sequentially, and it typically cannot revisit earlier data once it has moved on to the next task.

The phrase "without access to all historical data simultaneously" is the key constraint. If you could always train on all data seen so far, you would simply retrain the model on the growing combined dataset. Continual learning becomes a problem precisely because this option is unavailable, whether due to storage limits, privacy constraints, regulatory requirements, or simply the volume of data that accumulates over a long deployment lifetime.

The key tension is between two competing objectives:

• Plasticity: the ability to learn new information and adapt to new tasks
• Stability: the ability to retain previously learned knowledge

An ideal continual learner would be fully plastic (rapidly absorbing new information) and fully stable (never degrading on past tasks). In practice, these objectives conflict directly. Increasing plasticity means the model's parameters shift more aggressively toward new data, which naturally overwrites gradients from old data. Increasing stability means the model's parameters resist change, which slows learning on new tasks. This tradeoff is called the plasticity-stability dilemma, and it is the central challenge in continual learning.

The dilemma is not unique to machine learning. Cognitive science has grappled with the same tension for decades, trying to understand how the human brain acquires new information across a lifetime without erasing old memories. The neural network model of the brain was initially expected to suffer catastrophically from sequential learning, just as artificial neural networks do. The brain's solution involves a distributed memory architecture with distinct subsystems that operate at different learning rates, a design that has directly inspired some of the most effective continual learning algorithms.

Retraining from scratch is conceptually simple but practically expensive for several reasons:

• Compute cost: Training large models takes days or weeks on expensive hardware. Retraining for every incremental update is prohibitive at scale. A model with hundreds of billions of parameters trained on trillions of tokens costs millions of dollars to train once. Doing this every time new data arrives is economically untenable.
• Data retention: Retraining requires keeping all historical data. For large-scale language models, this means storing terabytes or petabytes of text indefinitely, which raises substantial storage, privacy, and regulatory costs. GDPR and similar regulations may require organizations to delete user data upon request, making indefinite retention legally problematic.
• Deployment latency: The gap between new data arriving and an updated model being deployed can stretch from hours to weeks when full retraining is required. For time-sensitive applications, such as financial news analysis or crisis response systems, this latency is operationally unacceptable.
• Catastrophic compute waste: Full retraining throws away all the gradient work done in previous training runs. Every iteration of a new training run spends compute rediscovering representations that the previous model already encoded. Incremental updates could in principle converge much faster if they could build on existing representations rather than relearning them from scratch.
• Biological implausibility: Humans do not forget how to ride a bicycle when learning to drive a car. They do not forget their native language when learning a second one. If the goal is human-like general intelligence, models should similarly accumulate knowledge across tasks rather than resetting whenever something new is learned.

These pressures motivate a different approach: learning from new data incrementally, updating only what needs to change, while preserving what was already learned. This is the continual learning program.

Let us formalize the continual learning problem. Suppose we have a sequence of tasks $\mathcal{T} = \{T_1, T_2, \ldots, T_N\}$, where each task $T_k$ is associated with a dataset $\mathcal{D}_k = \{(x_i^{(k)}, y_i^{(k)})\}_{i=1}^{n_k}$ drawn from distribution $p_k(x, y)$.

A model with parameters $\theta$ is trained sequentially. After training on task $T_k$, the model has parameters $\theta_k$. The continual learning objective is:

where:

• $\theta_N$: the model parameters after training on all $N$ tasks sequentially
• $\mathcal{L}_k(\theta_N)$: the loss on task $k$ evaluated with the final parameters, measuring how well the model has retained task $k$'s knowledge
• $N$: the total number of tasks encountered over the model's lifetime

The constraint is that when training on task $T_k$, the model typically has access only to $\mathcal{D}_k$, not to any $\mathcal{D}_j$ for $j < k$. This data constraint is what makes the problem hard. Without it, you could simply minimize the sum of losses jointly by training on all data simultaneously.

Notice what the objective asks for: it wants the final parameters $\theta_N$ to perform well on every task rather than only the last one. This is categorically different from the standard supervised learning objective, which only cares about performance on the current training distribution. The continual learning objective asks the model to remain competent across its entire history of tasks, even though the gradient updates during training on task $k$ will only ever see $\mathcal{D}_k$.

The gap between this objective and the data constraint is where all the difficulty lives. The optimizer has no signal from $\mathcal{D}_1, \ldots, \mathcal{D}_{k-1}$ when it updates $\theta$ during training on task $k$, so it has no direct reason to preserve performance on those earlier tasks. Any preservation must come from architectural constraints, memory mechanisms, or regularization that encode information about the past into the update rule itself.