Text Generation Applications › Prompt Engineering Patterns

Prompt engineering for text generation is broader than getting a first output; it requires designing prompts that reliably produce high-quality outputs across the distribution of inputs the system will encounter. The gap between a prompt that works for a demo and one that works in production is often large.

Zero-shot prompts describe the task without providing examples. Few-shot prompts include one or more example input-output pairs before the actual query. For instruction-tuned models, zero-shot prompting works well for common tasks that were well-represented in the fine-tuning data. For unusual formats, edge cases, or tasks that require a specific style not easily described in words, few-shot examples are more reliable.

The number of examples matters. One or two examples establish the pattern; more than five examples yield diminishing returns and use tokens that could be used for input context. Selecting diverse, high-quality examples is more important than maximizing example count. For production systems, maintaining a curated library of few-shot examples, selected based on performance on held-out validation sets, is standard practice.

For generation tasks that require planning or reasoning (e.g., writing a legal clause that must satisfy multiple conditions), asking the model to think through its approach before writing produces better outputs. The chain-of-thought pattern says: "First, outline the key points your response should cover. Then write the response." The planning step produces a structured intermediate representation that constrains the generation step, analogous to the hierarchical approach we discussed for long-form articles.

Chain-of-thought prompting also makes the model's reasoning visible for debugging. If the output is wrong, you can inspect the thinking step to see where the model's plan went wrong, rather than just observing a bad output with no insight into why.

Most production APIs separate the system prompt (which configures the model's behavior globally) from the user prompt (which specifies the current task). System prompts are the right place for stable configuration: persona, tone, general constraints, output format requirements, and safety rules. User prompts are the right place for variable input: the specific task, the input data, and per-request constraints.

Mixing these concerns causes problems. System prompts that include user-level constraints cause issues when those constraints need to vary across requests. User prompts that include persona configuration produce inconsistent behavior because the persona has to be re-established on every call. A strict separation between system and user prompt layers makes prompts more maintainable and easier to test.