GPT-3 › In-Context Learning: A Deeper Look

Now that we have covered the mechanics of how few-shot prompting works, it is worth going deeper into what in-context learning reveals about the nature of large language models. The discovery of in-context learning raised questions that go beyond engineering: what is happening computationally when a model reads a few examples and then generalizes to new inputs? How does this relate to what we call "learning" in more traditional settings? And what does it imply about the internal organization of knowledge in a large language model?

One of the most illuminating frameworks for understanding in-context learning comes from meta-learning, a branch of machine learning concerned with algorithms that can learn to learn. In meta-learning, a model is trained across many different tasks and learns to extract transferable strategies that help it adapt quickly to new tasks. The goal is not to learn any single task well but to develop the capacity for fast task acquisition.

GPT-3 was not explicitly trained with a meta-learning objective. Its training was simply next-token prediction on a diverse corpus of text. Yet the diversity of that corpus may have implicitly induced a form of meta-learning. When GPT-3 trained on a book about cooking, a paper on mathematics, a tutorial on programming, and a forum discussion about personal finance all at once, it was forced to maintain representations that were useful across all of these domains simultaneously. The attention mechanism, which can flexibly attend to any part of the context, is well-suited to this kind of flexible, context-dependent processing.

The key insight from this perspective is that in-context learning is not a special mode that activates when you provide demonstrations. It is the same process the model uses during ordinary language generation, applied to a context that happens to contain input-output examples. When GPT-3 reads the text "Translate English to French: sea otter => loutre de mer, plush girafe => girafe peluche, cheese => fromage", it is processing this as ordinary text, continuing the pattern it has learned to recognize across billions of documents. The few-shot format looks like many things the model has seen during pretraining: flashcard decks, translation dictionaries, worked examples in textbooks, and question-answer pairs. The model recognizes the pattern and applies the corresponding text-generation strategy.

This view makes a testable prediction: the more a task's format resembles text formats common in pretraining, the better in-context learning will work for that task. And this is broadly what researchers found. Translation tasks, which appear extensively in multilingual web text, work well. Mathematical proof tasks, which require formats and reasoning patterns rarely seen in web text, work poorly. Code completion tasks, which appear extensively in GitHub repositories that were likely included in or similar to GPT-3's training data, work better than you might expect for a language model.

A critical experiment in understanding in-context learning was conducted by Min et al. (2022), who systematically studied which information in few-shot demonstrations affects performance. They tested four variants of demonstrations:

1. Standard demonstrations: Correct input-output pairs in the correct format.
2. Random labels: Correct inputs paired with randomly assigned labels from the label set.
3. Random inputs: Inputs replaced with random words or sentences, paired with correct labels.
4. No demonstrations: Zero-shot baseline.

The surprising finding was that random labels (variant 2) performed nearly as well as standard demonstrations for many tasks, while providing much better performance than zero-shot. This strongly suggested that the labels themselves were not the primary source of benefit. Instead, the main contribution of demonstrations appeared to be: specifying the input distribution (showing what kinds of inputs to expect), specifying the output distribution (showing what kinds of outputs are expected), and specifying the input-output format (showing how inputs and outputs are structured relative to each other).

The semantics of the label mapping (which input class corresponds to which output label) contributed much less than expected. This result was initially controversial because it seemed to undermine the claim that in-context learning was learning the task from the examples. If the labels don't matter, the model isn't learning the input-to-output mapping from the demonstrations; it is doing something else.

Subsequent work refined this picture. While random labels hurt less than expected on simple tasks, they hurt more on complex tasks, especially those involving multiple-step reasoning or subtle semantic distinctions. The conclusion is that demonstrations serve multiple functions simultaneously, and different tasks rely on different aspects. Simple classification tasks benefit primarily from format information. Complex reasoning tasks benefit additionally from the actual input-output mappings in demonstrations.

However in-context learning works at a high level, it must be implemented through the model's actual computational primitives: the attention mechanism and the feed-forward networks. Understanding how attention supports in-context learning gives insight into both the capabilities and the limits of this approach.

Consider what happens when GPT-3 processes a few-shot prompt for sentiment classification. Early layers of the transformer build representations of individual tokens, incorporating local context. Middle layers begin to form representations of longer phrases and sentences. Later layers build higher-level semantic representations that encode the meaning and function of entire passages.

As the model processes the demonstration pairs, the attention mechanism creates associations between input texts and their corresponding labels. When the test input is processed, later-layer attention heads can attend back to demonstration inputs that are semantically similar to the test input. Through the value vectors, the associated labels influence the model's representation of the test input. The more demonstrations, and the more semantically relevant they are to the test input, the stronger these associations become.

Notice that this is not a perfect implementation of the task. The model's ability to find the right demonstrations to attend to depends on the quality of its representations, which depend on pretraining. If the test input is dissimilar to all demonstrations in the model's internal representation space, the attention mechanism cannot bridge the gap. This explains why semantically diverse demonstrations help and why demonstrations similar to the test input are particularly valuable.

The feed-forward networks play a complementary role. Research by Geva et al. (2021) found that feed-forward layers in transformers function somewhat like a key-value memory, where each neuron can be seen as storing a pattern and its associated knowledge. During pretraining, these memories are filled with associations learned from the training corpus. During in-context learning, the representations built by the attention mechanism serve as queries that retrieve relevant memories from the FFN. The final prediction combines the in-context signal from attention with the pretraining knowledge stored in the FFN weights.

This combination is what makes large models so capable in the few-shot setting. A small model can learn in-context signal, but it has limited stored knowledge to combine it with. A large model has both the capacity to learn from in-context demonstrations and a vast store of pretrained knowledge to draw on when interpreting and extending those demonstrations.