Text Generation Applications › The Generation Paradigm
Language AI HandbookAll chapters Text Generation Applications › The Generation ParadigmOne section · Sections in this chapter2 / 11 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
A large language model trained with causal language modeling learns one objective: predict the next token given all previous tokens. We covered this objective in detail in the chapter on Causal Language Modeling. What makes this deceptively powerful is that the same objective, applied at sufficient scale on sufficiently diverse text, produces a model that can perform nearly any text manipulation task when prompted correctly.
The generation paradigm has three essential ingredients:
- A pre-trained language model with broad world knowledge encoded in its parameters
- A prompt that frames the task, provides context, and shapes the output style
- A decoding strategy that selects tokens from the model's probability distribution
The chapters on Autoregressive Generation, Decoding Temperature, Top-k Sampling, Nucleus Sampling, and Repetition Penalties covered decoding strategies. Here we focus on the application layer: what you build on top of that generation machinery.
It is worth pausing to appreciate how unusual this situation is. Traditional software requires explicit programming: a developer writes rules, a compiler translates them to machine code, and the system executes those rules exactly. A text generation system inverts this entirely. You describe the desired behavior in natural language, and the system produces outputs that approximate it. The approximation is almost never perfect, which is why evaluation, constraint checking, and quality pipelines are not optional extras in production systems. They are the engineering layer that compensates for probabilistic behavior.
From Completion to Instruction Following
Early GPT-style models performed completion: given a prefix, continue the text. This works for generation tasks if you carefully craft a prefix that demonstrates the pattern, but it requires users to think like the model. Effective few-shot prompting for completion models required providing two or three examples of the desired input-output pattern before the actual query, so that the model could infer the task from the distributional signal of the examples. This is called few-shot in-context learning, which we explored in the In-Context Learning chapter.
Instruction-tuned models, as covered in the Instruction Tuning chapters, shift the interface. You describe what you want ("Write a product description for a wireless keyboard targeting remote workers") and the model complies. This distinction matters practically because it determines how you write prompts and how reliably the model follows task constraints. Completion models require prompt engineering as a craft; instruction-following models require it as a discipline, but the craft is more forgiving.
The instruction-following interface also matters for how constraints are communicated. A completion model implicitly infers constraints from the examples you provide; an instruction-following model reads constraints you state explicitly. This makes constraint specification more reliable and more debuggable. When the model violates a constraint, you can inspect your prompt and identify whether the constraint was stated clearly, whether conflicting instructions existed, or whether the model simply failed to follow an unambiguous instruction.
A completion model continues text by matching distributional patterns. An instruction-following model has been fine-tuned to treat user messages as task specifications and produce task-appropriate outputs. Most modern APIs expose instruction-following models. In practice, an instruction-following model is a completion model fine-tuned with supervised learning on instruction-response pairs and reinforcement learning from human feedback (RLHF), which nudges it to comply with user intent rather than merely continue likely text.
The Role of Scale
Not all text generation capabilities emerge at the same model scale. The ability to generate grammatically correct text appears at relatively small scales; the ability to maintain factual consistency across a 500-word document requires substantially more capacity; the ability to follow complex multi-constraint instructions (generate a 200-word summary in the voice of a 19th-century novelist without using the word "the") requires very large models. This graduation means that application requirements drive model selection, and cost-performance trade-offs are real. Matching the model to the task is one of the most important engineering decisions in production text generation systems.
Small models (under 7 billion parameters) are fast and cheap but struggle with complex instruction following and factual accuracy. Medium models (7 to 70 billion parameters) balance capability and cost well for most content generation tasks. Very large models (70 billion parameters and above) are justified for high-stakes tasks where accuracy and nuance matter more than cost. Distillation techniques, which transfer knowledge from large models into small ones, are increasingly practical for specific tasks, as we will discuss in the Model Compression chapters.