Backpropagation Through Time › Backpropagation Through Time

Training a feedforward network is conceptually straightforward: run a forward pass through a fixed-depth computation graph, then run backpropagation to compute gradients with respect to every parameter. The graph has a clear start and end, and each layer gets visited exactly once in each direction. Every parameter is used at most once per forward pass, so its gradient is computed from a single path through the network.

Recurrent neural networks break that assumption. As we explored in the RNN Architecture chapter, an RNN processes a sequence one step at a time, feeding its hidden state forward from step to step. The same weight matrix $\mathbf{W}_{hh}$ gets applied at every timestep, which means a single parameter participates in every step of the computation. At step 1, it shapes the first hidden state. At step 2, it shapes the second. At step 50, it shapes the fiftieth. When you ask: "How should I adjust $\mathbf{W}_{hh}$ to reduce the loss?", the answer requires aggregating gradient contributions from all fifty steps simultaneously.

This is the problem that backpropagation through time (BPTT) solves. It is not a fundamentally new algorithm. BPTT is the same chain rule that powers all neural network training, applied to a computation graph that extends across time. The key insight is that by "unrolling" the RNN, you convert the looping recurrent structure into a deep but regular feedforward graph, and then you apply backpropagation to that unrolled graph exactly as you would to any other network.

The result is an elegant algorithm that correctly computes gradients for all shared parameters across all timesteps. But the unrolling operation exposes a fundamental tension: the deeper the unrolled graph (the longer the sequence), the longer the gradient paths, and the more severe the numerical problems that arise from multiplying many matrices together. BPTT's structure creates challenges that feedforward backprop never faces, chief among them the vanishing and exploding gradient problems we will examine closely in this chapter and the next.

This chapter explains how BPTT works from first principles, derives the gradient equations that flow backward through time, covers truncated BPTT for practical training on long sequences, analyzes the computational and memory costs in detail, and shows how to implement BPTT correctly in PyTorch. By the end, you will understand how the algorithm works, why it behaves the way it does on long sequences, and what practitioners have learned to do about it.