Long-Form Generation › Outline-Based Generation
Language AI HandbookAll chapters Long-Form Generation › Outline-Based GenerationOne section · Sections in this chapter3 / 10 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
The most intuitive approach to long-form generation mirrors how human writers work: plan first, then write. Outline-based generation separates the task into two stages. In the first stage, the model generates an outline or plan. In the second stage, it generates the actual content, using the outline as a fixed structural guide.
This two-stage approach was explored systematically in systems like ROME (Rashkin et al., 2020) and later in commercial applications of instruction-tuned models. The core finding is consistent: models with an explicit plan produce significantly more coherent long-form text than models that generate free-form without structural constraints. The plan does not need to be perfect; even a rough outline improves the consistency and completeness of the final document.
Why Outlines Help
An outline is a persistent representation of document structure that the model can reference throughout generation. Rather than holding the full structure implicitly in its weights and context, the model has an explicit, compact summary of what the document contains and where it is going. This provides several benefits.
First, outlines prevent drift. If the outline specifies that section 3 covers "model evaluation metrics," the model cannot easily wander into discussing deployment infrastructure, because the next section header provides a constant anchor. The anchor does not need to be elaborate; even a simple list of section titles significantly constrains where the model can go.
Second, outlines enable better global planning. A model that knows the full document structure can use early sections to set up content that pays off later, creating callbacks and building toward a conclusion. Without an outline, the model cannot see far enough ahead to make strategic choices about what to introduce early versus what to save for later.
Third, outlines make generation controllable. Users can inspect and edit the outline before content generation begins, catching structural problems early. If the model proposes six sections but the user needs eight, or if the proposed order does not match the intended argument, these are easy to fix at the outline stage. They are much harder to fix after 5,000 words have been generated.
Fourth, outlines provide a natural chunking mechanism. Rather than asking a model to generate an entire document in one pass (which it cannot do reliably), the outline breaks the task into section-sized units, each of which is within the model's comfortable generation range.
Outline Generation
Generating good outlines is itself a challenging task. A well-formed outline should be:
- Complete: covering all major topics the document needs to address
- Non-redundant: avoiding sections that overlap substantially in content
- Logically ordered: following a sequence that builds understanding progressively
- Appropriately granular: neither too broad (useless) nor too detailed (constraining)
The tension between granularity and flexibility is real. An outline that specifies ten subsections per section leaves the model little room to follow the natural contours of the topic. An outline that specifies only top-level section titles may not provide enough guidance to prevent drift within sections. A useful rule of thumb is to specify outlines at the level that identifies the purpose of each section, not the content of each paragraph.
In practice, outline generation is prompted differently than content generation. The model is given the topic, the target audience, and the intended length, and asked to produce a hierarchical structure before any actual content. Here is a simple example of how such a call is structured:
import openai
client = openai.OpenAI()
def generate_outline(topic: str, target_length: str, audience: str) -> str:
"""Generate a hierarchical outline for a long-form document."""
prompt = f"""You are a professional writer and content strategist.
Topic: {topic}
Target length: {target_length}
Audience: {audience}
Generate a detailed hierarchical outline for this document. Include:
- A clear introduction section
- 4-6 main sections covering the key aspects
- 2-4 subsections per main section where appropriate
- A conclusion section
Format as a numbered outline with indented subsections."""
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": prompt}],
temperature=0.7,
max_tokens=1000,
)
return response.choices[0].message.contentCalling generate_outline("Long-Form Generation in NLP", "5,000 words", "CS students") produces a structured plan like the following:
1. Introduction: Why Long-Form Generation Matters 1.1 The gap between sentences and documents 1.2 Current model capabilities and limitations 1.3 Chapter overview 2. Context Window Constraints 2.1 Fixed-length attention windows 2.2 The lost-in-the-middle phenomenon 2.3 Strategies for working within limits 3. Outline-Based Generation 3.1 Separating planning from writing 3.2 Generating effective outlines 3.3 Conditional content generation from outlines 4. Hierarchical Generation 4.1 Decomposing documents into subtasks 4.2 Top-down vs. bottom-up approaches 4.3 Maintaining consistency across levels 5. Coherence Maintenance 5.1 Local vs. global coherence 5.2 Entity tracking 5.3 Consistency checking 6. Evaluation 6.1 Reference-based metrics 6.2 Coherence scoring 6.3 Human evaluation protocols 7. Conclusion 7.1 Summary of key techniques 7.2 Open challenges
A well-structured outline like this gives the content generation phase clear targets for each section. Notice that the outline itself is compact: roughly 200 tokens, which is negligible compared to the full document. This compactness matters. The outline can be included in every section's generation prompt without consuming significant context budget.
Conditional Generation from Outlines
Once an outline exists, content is generated section by section, with each section prompt including:
- The full outline (for global context)
- The specific section being generated
- The previously generated sections (for local context and continuity)
This approach ensures the model always knows both where the document is going and what has already been written. The prompt structure deliberately places the outline first, exploiting the observation that models recall information near the beginning of context more reliably.
The function for generating a single section looks like this:
def generate_section(
outline: str,
section_title: str,
section_index: int,
previous_sections: list,
words_per_section: int = 400,
) -> str:
"""Generate a single section, conditioned on the full outline and prior sections."""
# Build context from previous sections (truncate if needed)
prior_context = "\n\n".join(previous_sections[-2:]) # Last 2 sections
prompt = f"""You are writing a long-form document. Here is the full outline:
{outline}
Here is the content written so far (last 2 sections):
{prior_context}
Now write the content for section {section_index}: "{section_title}"
Requirements:
- Approximately {words_per_section} words
- Connect naturally to what was written before
- Use the outline to guide what this section should cover
- Do not repeat information from previous sections"""
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": prompt}],
temperature=0.7,
max_tokens=600,
)
return response.choices[0].message.contentSection generation parameters: section_title: Context Window Constraints section_index: 2 words_per_section: 400 prior_context_sections: 2 The model receives: - Full outline (compact, ~200 tokens) - Last 2 generated sections (continuity context) - Target section title and length - Constraint: no repetition from prior sections
By conditioning on the outline throughout, the model maintains document-level coherence even as it generates section by section. An important detail is the choice to include only the last two sections of prior context, rather than all prior sections. Including all prior content would quickly exhaust the context window; including only recent sections provides enough continuity for local coherence without sacrificing token budget.
Recursive Outline Expansion
For very long documents, outlines can themselves be expanded hierarchically. A top-level outline of 6 sections might be expanded into a more detailed outline for each section, which is then expanded into paragraph-level plans before content generation begins. This multi-level planning reduces the cognitive burden on each generation step, since each step has clear, narrow scope.
The expansion process works recursively. Given a section title and its purpose (from the top-level outline), the model generates a subsection-level outline for that section. Then, given a subsection title, it generates paragraph-level summaries. By the time actual prose generation begins, every paragraph has a well-defined purpose, and the prose generation step is reduced to the narrow task of expressing a known idea in fluent language.
This mirrors how skilled human writers work. Professional authors often produce detailed synopses before writing a single word of prose, precisely because planning and writing are cognitively different tasks that interfere with each other when done simultaneously.