GPT-3 › Prompt Engineering Principles

The success of in-context learning spawned a new discipline: prompt engineering. Researchers discovered that how you format prompts dramatically affects performance. This was simultaneously exciting and troubling: exciting because it meant that performance could be improved without any model changes, troubling because it made deployment unreliable. A prompt that worked beautifully on test cases might fail unexpectedly on edge cases that differed in subtle ways.

The practical consequence was a new kind of expertise. Organizations deploying GPT-3 needed people who understood how to write effective prompts, test them systematically, and iterate when they failed. Prompt engineering became a recognized skill, and eventually a job title. The discipline drew on insights from cognitive science (how to communicate instructions clearly), information retrieval (how to retrieve relevant context), and software engineering (how to test and validate behavior systematically).

Small changes in prompt format can yield large performance differences:

Research showed that the conversational format often performs best for GPT-3, likely because its training data contains many examples of conversational task-solving. The optimal format varies by model and task, requiring empirical experimentation. The simple format is terse and may not trigger the right pretraining associations. The instructional format is explicit but somewhat mechanical. The structured format uses markdown-style formatting that may or may not match the model's internal representations. The conversational format frames the task in the way you might address a human expert, which often aligns well with the task-solving patterns the model has internalized from instructional web content.

Notice that the conversational format does something the others don't: it establishes a role for the model ("you are a sentiment analysis expert") before asking it to perform. This role assignment can significantly influence the style and accuracy of responses. When the model is told it is an expert, it tends to produce more confident, technically precise outputs. When it is not given a role, outputs can be more hedged and generic. Role assignment is one of the most reliable prompt engineering techniques.

The choice and order of few-shot examples strongly affects performance:

• Diversity: Examples should cover the range of possible inputs and outputs. If you are doing sentiment classification and all your examples are strongly positive or strongly negative, the model may struggle with neutral or mixed inputs.
• Similarity: Examples semantically similar to the test input often help more. If your test case is a hotel review, hotel review examples will generally outperform product review examples, even for the same classification task.
• Recency: Examples closer to the end of the prompt have stronger influence due to recency effects in attention. The model attends more strongly to nearby context than to distant context.
• Balance: For classification, examples should be balanced across classes. An imbalanced prompt where 3 out of 4 examples are Positive will bias the model toward predicting Positive, regardless of the test input.

In practice, this suggests a concrete strategy: if you have one example that is much more similar to your expected test inputs than the others, place it last. The recency effect means the model will weight it most heavily, and the semantic similarity between the last example and the test input will further amplify its influence. Conversely, if you have an outlier example that you want to include for coverage but don't want to dominate predictions, place it early in the prompt.

A surprising finding was that the actual labels in few-shot examples don't always matter. In some experiments, randomly assigning labels (positive reviews labeled "Negative" and vice versa) still improved performance over zero-shot, suggesting the model was learning format rather than task semantics.

However, for more complex tasks, correct labels improve performance considerably. The model can use both the format pattern and the input-output mapping. A useful rule of thumb: for simple classification tasks where the labels are self-explanatory (Positive/Negative, True/False), format information dominates. For complex tasks where the input-output mapping is not obvious, correct labels are critical because they provide the only signal about what the task requires.

This finding has a practical implication: you can sometimes bootstrap a few-shot prompt without labeled data, using the correct format with placeholder outputs to establish the input-output structure, then relying on the model's pretraining to supply the semantics. This is particularly useful when labeled data is expensive or unavailable, though it requires careful validation to ensure the model's pretraining assumptions match your intended task.