Encoder-Decoder Framework › Encoder Architecture

The encoder is the "reader" half of the system. Its job is to process an arbitrary-length input and distill it into a compact representation that the decoder can work from. Before diving into the mathematics, it helps to build intuition about what this compression process looks like.

Think of the encoder as a reader who is given a passage and asked to write a summary on a notecard of fixed size. After reading each sentence, the reader updates their mental model of what they have read so far. By the end of the passage, the notecard should contain everything important about the passage, but compressed into whatever fits. Early sentences may not make it onto the notecard if they were less important, or they may be represented implicitly through their effect on the reader's evolving understanding. This is exactly what the LSTM encoder does: its hidden state at each step is an evolving compressed summary, and the final state is the notecard passed to the decoder.

The encoder processes the source sequence $x_1, x_2, \ldots, x_T$ and produces a sequence of hidden states $h_1, h_2, \ldots, h_T$. At each timestep $t$, the encoder computes a new hidden state based on the current input token and the previous hidden state:

$$h_t = f_{\text{enc}}(x_t, h_{t-1})$$

where:

• $h_t \in \mathbb{R}^d$ is the encoder hidden state at time $t$
• $x_t$ is the embedding of the $t$-th input token
• $f_{\text{enc}}$ is the encoder recurrent cell (LSTM, GRU, or vanilla RNN)
• $d$ is the hidden size, a hyperparameter, typically 256 to 1024

After processing all $T$ input tokens, the context vector $c$ is taken as the final encoder hidden state:

$$c = h_T$$

where:

• $c \in \mathbb{R}^d$ is the context vector
• $h_T$ is the encoder hidden state after processing the last input token

For LSTM encoders, the encoder produces both a hidden state $h_T$ and a cell state $s_T$, and both are passed to the decoder. This pair $(h_T, s_T)$ together forms the context.

The encoder starts with a zero hidden state $h_0 = \mathbf{0}$ and accumulates information about the input as it reads token by token from left to right. By the time it reaches the end of the sequence, the hidden state theoretically contains a summary of everything the encoder has seen. In practice, vanilla RNNs struggle to retain information from the beginning of long sequences due to vanishing gradients, which is why LSTMs and GRUs are preferred for the encoder.

Before the encoder processes any tokens, the source vocabulary must be mapped to dense vectors using an embedding layer. If the source vocabulary has size $V_{\text{src}}$ and we use embedding dimension $d_e$, the embedding matrix $E_{\text{src}} \in \mathbb{R}^{V_{\text{src}} \times d_e}$ maps each integer token index to a continuous vector. These embeddings are learned jointly with the rest of the model during training.

The encoder receives these embedded vectors as input at each timestep, not the raw integer indices. The embedding layer is a required component: it performs dimensionality reduction from a one-hot space of size $V_{\text{src}}$ (which may be 30,000 or more tokens) to a dense space of size $d_e$ (typically 128 to 512), and it learns to position semantically related words close together in that space. A word like "run" and "sprint" will end up with similar embeddings after training on parallel corpora, because they appear in similar translation contexts.

Embedding dimension and hidden dimension do not need to match, though they often do in practice. If they differ, a linear projection is typically inserted between the embedding layer and the recurrent cell to map the embedding into the recurrent input space.

A standard encoder processes the input strictly left to right, meaning that the hidden state at position $t$ only incorporates information from tokens $x_1$ through $x_t$. The hidden state at position $T$ therefore has the most information, but the hidden state at position 1 has almost none about the rest of the sequence. When we use only the final state as the context vector, we are relying on the LSTM's ability to remember all relevant early tokens, which degrades with sequence length.

Bidirectional encoders address this by running two separate recurrent layers over the input: one in the forward direction (left to right) and one in the backward direction (right to left). The forward layer produces states $\overrightarrow{h}_1, \ldots, \overrightarrow{h}_T$ and the backward layer produces states $\overleftarrow{h}_1, \ldots, \overleftarrow{h}_T$. At each position, the two states are concatenated to give a full representation that incorporates context from both directions:

$$h_t = [\overrightarrow{h}_t; \overleftarrow{h}_t] \in \mathbb{R}^{2d}$$

For the context vector in the basic encoder-decoder framework, we typically concatenate the final forward state and the initial (from the backward direction) backward state:

$$c = [\overrightarrow{h}_T; \overleftarrow{h}_1]$$

This gives the context vector access to the full forward pass summary and the full backward pass summary simultaneously. This provides a richer, more symmetric encoding of the source. In practice, bidirectional encoders consistently outperform unidirectional ones, especially for tasks where word order matters or where early tokens are important for understanding the sentence meaning.

The main cost of bidirectionality is that the context vector doubles in size (from $d$ to $2d$), which means the decoder must also use hidden states of size $2d$, or a linear projection is needed to bring the context back to size $d$ before feeding it to the decoder.

Just as we can stack LSTM layers in a standard RNN, we can stack them in the encoder. With $L$ layers, the first layer takes the embedded tokens as input, and each subsequent layer takes the hidden states of the layer below as input:

$$h_t^{(l)} = f_{\text{enc}}^{(l)}\!\left(h_t^{(l-1)}, h_{t-1}^{(l)}\right)$$

The final context vector is taken from the topmost layer: $c = h_T^{(L)}$. In the Sutskever et al. 2014 paper, a 4-layer LSTM encoder with reversed source input was used, and the depth was an important contributor to the model's performance. Deep encoders can learn hierarchical representations: lower layers might capture syntactic structure while higher layers capture semantic content, analogous to how convolutional networks build from edge detectors to object detectors.

Deeper encoders require correspondingly deeper decoders with matching layer counts, and the hidden state from each encoder layer is used to initialize the corresponding decoder layer. This ensures that information at different levels of abstraction is properly propagated across the bottleneck.