Architecture Methods › Progressive Networks

Progressive networks, introduced by Rusu et al. in 2016 within the context of reinforcement learning transfer, represent the cleanest possible architecture solution to forgetting: every time a new task arrives, you freeze the entire existing network and instantiate a new column. The new column is a fresh network of the same depth as the existing ones. It receives lateral connections from every previous column at every layer, allowing it to draw on previously learned features while leaving old parameters completely untouched.

The motivation behind progressive networks was initially about transfer in reinforcement learning rather than continual learning per se. Rusu and colleagues were training a sequence of Atari games and wanted each new game to benefit from representations learned on prior games, while simultaneously ensuring that training on the new game would not degrade performance on earlier ones. The standard solution at the time was fine-tuning, which provided transfer but destroyed old performance. Progressive networks offered a way to get both benefits, at the cost of growing the network with each new domain.

A progressive network with $K$ tasks consists of $K$ columns. Column $k$ has parameters $\theta^{(k)}$ and produces layer activations $h_i^{(k)}$ at layer $i$. When training on task $k$, all columns $1$ through $k-1$ are frozen. The forward pass through column $k$ at layer $i$ computes:

$$h_i^{(k)} = f\!\left( W_i^{(k)} h_{i-1}^{(k)} + \sum_{j < k} U_i^{(k:j)} h_{i-1}^{(j)} \right)$$

where:

• $W_i^{(k)}$: the weight matrix for layer $i$ of column $k$, trained on the current task
• $U_i^{(k:j)}$: the lateral connection matrix from layer $i-1$ of column $j$ to layer $i$ of column $k$
• $h_{i-1}^{(j)}$: the activation from layer $i-1$ of a previous column $j$
• $f$: a non-linear activation function (typically ReLU)

The sum over all $j < k$ means that column $k$ can attend to the layer-$(i-1)$ activations of every preceding column. This is a multi-source feature reuse mechanism: the new column sees its own evolving representations and the frozen representations that already encode prior task knowledge.

The lateral connections $U_i^{(k:j)}$ are initialized to zero and trained alongside $W_i^{(k)}$. Initializing them to zero means the new column starts by computing its activations independently, and the lateral pathways open up only as training on task $k$ finds them useful. This is an important design choice: it prevents prior columns from immediately dominating the new column's representation, allowing the new task to establish its own features first before selectively borrowing from what came before.

In the original paper, the lateral connections were implemented using adapter modules rather than raw weight matrices. Specifically, the lateral from column $j$ to column $k$ at layer $i$ passes through a low-rank projection that compresses the hidden-dim-sized activation of column $j$ before injecting it into column $k$. This compression serves two purposes. First, it reduces the number of parameters in the lateral connections, which would otherwise grow quadratically with the number of columns. Second, it allows the projection to learn a kind of relevance filter: the low-rank adapter can selectively amplify the dimensions of the prior column's representation that are useful for the current task while suppressing irrelevant ones.

Freezing prior columns provides an absolute guarantee: no matter what happens during training on task $k$, the outputs of columns $1$ through $k-1$ for any old task input are identical to what they were when those tasks were learned. There is no forgetting in the traditional sense because there is no parameter sharing across time.

This guarantee is fundamentally different from the probabilistic protection offered by regularization. EWC and its relatives use a penalty term to make it costly to move important weights, but the penalty is finite. Sufficiently strong gradients from new tasks can still overwrite old representations, especially when the new task requires moving in a direction that happens to coincide with a direction important for old tasks. Progressive networks eliminate this fragility entirely by making old parameters literally unmodifiable.

The cost is that the number of parameters grows linearly with the number of tasks. After $K$ tasks, you have $K$ columns plus the lateral connection matrices between them. The lateral connections alone total $O(K^2)$ matrices (one for each column-pair at each layer), though in practice they are projected through a lower-rank adapter to keep this manageable. The storage footprint after 100 tasks exceeds 100 times the single-task footprint because of the lateral adapters.

One of the underappreciated strengths of progressive networks is that they implement a structured form of forward transfer. When column $k$ trains, its lateral connections to all prior columns are free to grow from zero to whatever values are helpful. This means the model can actively decide which prior columns' features to borrow at each layer.

Empirically, Rusu et al. observed that the lateral connections tend to be denser (larger weights) at later layers than earlier layers. This makes intuitive sense: early layers in vision networks learn low-level detectors such as edge detectors and color filters that tend to be universal across tasks, so reusing them is natural and expected. Deeper layers encode more task-specific abstractions that may or may not be transferable. When two tasks are semantically related, you expect denser lateral connections in deeper layers, and that is precisely what was observed when training on Atari games with similar visual statistics.

The forward transfer in progressive networks is automatic and unsupervised in the sense that no explicit signal tells the model which tasks are similar. The lateral connections learn the degree of sharing purely from gradient descent on the new task's objective. If borrowing from an old column helps minimize the new task's loss, the lateral connections strengthen. If it does not, they stay near zero.

The linear parameter growth is the defining limitation of progressive networks. If each column has $N$ parameters, after 100 tasks you have 100 columns and roughly $100N$ parameters to store and run inference through. This is acceptable for tens of tasks in constrained domains like robotics or game-playing, but becomes impractical for continual learning across hundreds or thousands of tasks.

A subtler issue is that progressive networks require knowing task identity at inference time. To produce a prediction for an input from task $k$, you run the input through columns $1$ through $k$ and read off column $k$'s output. Without knowing which column to read from, the architecture does not naturally produce a single prediction. This is a general limitation of architecture-based approaches: they often assume task-specific routing at inference time, an assumption that is realistic for clearly delineated task sequences but breaks down in natural open-ended deployment where task boundaries are fuzzy or unknown.

A third limitation is that the architecture assumes all tasks use the same input format and the same output space, or at least a compatible one. If task 1 is image classification and task 5 is depth estimation, you cannot straightforwardly connect them through shared lateral connections. In practice, progressive networks have been most successful in homogeneous settings where all tasks are variations of the same problem type.