GPT-3 › How In-Context Learning Works

The mechanism behind in-context learning remained mysterious. How can a model "learn" a task without updating its weights? Several hypotheses emerged from subsequent research, each offering a different lens on what is happening during the forward pass when a model encounters few-shot demonstrations.

The mechanism matters in practice. If in-context learning works through task recognition, then the examples mainly activate pre-stored knowledge, and their format matters more than their quality. If it works through implicit fine-tuning, then the examples need to provide correct statistical signal about the mapping from inputs to outputs. If it works through Bayesian inference, then more examples are always helpful because they sharpen the posterior. Different mechanisms have different implications for how to engineer effective prompts.

One theory suggests that pretraining exposes the model to many implicit task formats. When given few-shot examples, GPT-3 recognizes the pattern from pretraining and activates the appropriate "circuit" for that task. The examples don't teach the model something new; they help it identify which of its existing capabilities to apply.

The evidence for this view comes from a striking experimental result: in several studies, randomizing the labels in few-shot examples (assigning "Positive" to negative reviews and vice versa) still improved performance over zero-shot. If the examples were teaching the input-label relation, scrambled labels should hurt performance significantly. That they don't suggests the main benefit of examples is format information, not label information. The model sees the input/output format and activates the corresponding task processing circuit, regardless of whether the specific labels are correct.

The analysis reveals structural patterns that a model might use to identify the task. With only two unique outputs (Positive, Negative) and short single-word responses, the format clearly signals a binary classification task. The model may recognize this pattern from similar structures encountered during pretraining, such as review datasets or labeled examples in web text. Think of it like a key fitting a lock: the format of the prompt matches patterns the model has seen before, triggering the relevant internal computation.

The limitation of this hypothesis is that it cannot explain all in-context learning behavior. For tasks absent from the pretraining data, the task recognition account predicts that in-context learning should fail entirely. But GPT-3 shows some ability to adapt to novel input-output formats, suggesting that task recognition alone is insufficient.

Another perspective views the forward pass through GPT-3's 96 layers as an implicit optimization process. The key insight is that transformer layers can implement gradient-descent-like updates on the representations. Consider a single attention layer operating on demonstrations $\{(x_i, y_i)\}$ followed by a test input $x_{\text{test}}$. The attention mechanism computes:

$$h_{\text{test}} = h_{\text{test}}^{(0)} + \sum_{i=1}^{k} \alpha_i \cdot v_i$$

where:

• $h_{\text{test}}^{(0)}$: the initial representation of the test input before attention
• $\alpha_i$: the attention weight from the test input to demonstration $i$
• $v_i$: the value vector derived from demonstration $i$
• $k$: the number of demonstrations in context

This weighted sum resembles a gradient update: the model adjusts its representation of the test input based on the demonstrations. The attention weights $\alpha_i$ function like step sizes, and the value vectors $v_i$ function like gradient directions. With 96 layers, the model has large "depth" to iteratively refine this adaptation. Research by Akyürek et al. (2022) showed that linear attention transformers can implement gradient descent in-context. This provides a theoretical foundation for this hypothesis. Further work found that in-context learning and explicit fine-tuning produce similar representational changes in the model's hidden states, even though one updates weights and the other doesn't.

The implicit fine-tuning hypothesis has an important implication: the quality of examples matters, not just their format. If the forward pass is implementing something like gradient descent on the demonstrations, then wrong labels should hurt, correct labels should help, and more examples should provide better gradient estimates, all consistent with observed behavior for most tasks.

A third hypothesis frames in-context learning as Bayesian inference over tasks. The model implicitly maintains a prior distribution $P(\text{task})$ over possible tasks from pretraining. Each demonstration $(x_i, y_i)$ updates this posterior according to Bayes' rule:

$$P(\text{task} \mid x_1, y_1, \ldots, x_k, y_k) \propto P(y_1, \ldots, y_k \mid x_1, \ldots, x_k, \text{task}) \cdot P(\text{task})$$

where:

• $P(\text{task})$: the prior probability of each task, learned during pretraining from exposure to diverse text formats
• $P(y_i \mid x_i, \text{task})$: the likelihood of output $y_i$ given input $x_i$ under a specific task interpretation
• $P(\text{task} \mid \ldots)$: the posterior probability of each task after observing demonstrations

By the time the model reaches the test input, it has narrowed down the task distribution sufficiently to make a confident prediction. This view explains why more demonstrations help: each one provides additional evidence that sharpens the posterior. It also explains why demonstrations with correct labels help more than scrambled ones for complex tasks: the likelihood term $P(y_i \mid x_i, \text{task})$ is only informative when the labels are accurate.

The Bayesian view predicts that performance should improve monotonically with more examples, which is roughly consistent with observations. It also predicts that unusual or contradictory demonstrations should hurt performance by introducing conflicting evidence, which is also consistent with empirical findings about prompt quality.

In practice, the true mechanism is likely a blend of all three hypotheses. The model uses format cues to identify the task type (task recognition), adjusts its representations through attention to align with the demonstration distribution (implicit fine-tuning), and these adjustments are consistent with what a Bayesian reasoner would do when updating over task hypotheses (Bayesian inference). The three accounts describe different levels of analysis: the computational level (what is being computed), the algorithmic level (how it is computed), and the representational level (what data structures are used).