Backpropagation Through Time › Summary

Backpropagation through time trains recurrent neural networks by unrolling the recurrent computation graph into a feedforward graph and applying standard backpropagation through it. The core ideas are:

• Unrolling converts recurrence to depth: An RNN over a sequence of length $T$ becomes a feedforward network with $T$ layers for the purpose of gradient computation, with all layers sharing the same weight matrices.
• Gradient accumulation: Because weights are shared across all timesteps, the gradient for any parameter is the sum of contributions from every timestep where that parameter participated.
• The backward error signal: The $\boldsymbol{\delta}_t$ recurrence propagates gradient information from later timesteps to earlier ones, accumulating contributions at each step.
• Truncated BPTT: Dividing sequences into chunks and using detach() at chunk boundaries bounds memory and computation while preserving the ability to carry forward state information.
• Gradient decay: Vanilla RNNs suffer from vanishing gradients when trained with BPTT on long sequences because the product of Jacobians across many timesteps tends toward zero. This motivates the gated architectures in the following chapters.
• Sequential computation: The strict timestep-by-timestep dependency in RNN forward passes limits hardware utilization and makes RNNs slower to train than architectures that can parallelize across positions, which was a key motivation for the development of the transformer.

The next chapter examines the vanishing gradient problem in detail, analyzing exactly why the product of Jacobians causes gradient signal to decay and what that means for what vanilla RNNs can and cannot learn.