GRU Architecture › Key Parameters

When working with GRUs in PyTorch (nn.GRU), several parameters significantly impact model behavior:

• hidden_size: The dimensionality of the hidden state vector. Larger values (256-1024) provide more memory capacity but increase computation. Start with 128-256 for most tasks and scale up if the model underfits.
• num_layers: Number of stacked GRU layers. Deeper networks (2-3 layers) can learn hierarchical patterns but are harder to train. Single-layer GRUs often suffice for moderate-complexity tasks.
• batch_first: When True, input tensors have shape (batch, seq_len, features). When False (default), shape is (seq_len, batch, features). Using batch_first=True aligns with common data loading patterns.
• dropout: Dropout probability applied between GRU layers (only active when num_layers > 1). Values of 0.1-0.3 help prevent overfitting. Has no effect on single-layer GRUs.
• bidirectional: When True, runs the GRU in both forward and backward directions, doubling the output hidden size. Useful for classification tasks where future context matters, but not applicable for autoregressive generation.
• input_size: The number of features in each input timestep. Must match your data's feature dimension exactly.