Michael Brenndoerfer | Data & AI, Private Equity, Technology

Michael Brenndoerfer

RAG Prompt Engineering: Context Placement & Citation Strategies › Code Implementation: Building a RAG Prompt Template

Michael BrenndoerferJanuary 31, 20266 min read

Part 44 of 67Retrieval-Augmented Generation

Chapter 344RAG Prompt Engineering: Context Placement & Citation Strategies › Code Implementation: Building a RAG Prompt Template

Reading section 7 of 11 · Code Implementation: Building a RAG Prompt Template · 6 minSection 7 of 11 · 6 min

Section 7 of 11 · 6 min read

RAG Prompt Engineering: Context Placement & Citation Strategies › Code Implementation: Building a RAG Prompt TemplateOne section · Sections in this chapter7 / 11 ↓

Sections in this chapter11

Reading tools Definitions & inline links

Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.

Article links

Make inline references clickable

Let's implement a flexible RAG prompt construction system that incorporates the principles we have discussed throughout this chapter.

In[9]:

Code

from dataclasses import dataclass
from typing import Any, Dict


@dataclass
class RetrievedDocument:
    id: str
    content: str
    score: float
    metadata: Dict[str, Any]

    def format_xml(self, max_length: int = 1000) -> str:
        truncated = self.content[:max_length]
        if len(self.content) > max_length:
            truncated += "..."
        meta_attrs = " ".join([f'{k}="{v}"' for k, v in self.metadata.items()])
        return f'<source id="{self.id}" relevance="{self.score:.3f}" {meta_attrs}>\n{truncated}\n</source>'

We define a RetrievedDocument class to hold our retrieved content with metadata. The format_xml method creates structured XML representations with truncation. The relevance score is included in the XML attributes so that the model has explicit access to the retrieval confidence for each document, which it can use when deciding how much weight to give each source.

In[10]:

Code

from typing import Dict, List


class RAGPromptBuilder:
    def __init__(self, max_context_tokens: int = 4000):
        self.max_context_tokens = max_context_tokens
        self.system_template = """You are a precise research assistant. Answer using ONLY the provided sources.

INSTRUCTIONS:
1. Cite every factual claim with [source_id] immediately after the claim
2. If multiple sources support a claim, cite all: [id1][id2]
3. If sources are insufficient, state: "The provided sources do not contain sufficient information."
4. When sources disagree, present all viewpoints with citations
5. Prioritize more recent sources when dates differ

AVAILABLE SOURCES:
{sources}

Answer the following question based strictly on the sources above."""

    def order_documents(
        self, docs: List[RetrievedDocument], strategy: str = "relevance"
    ) -> List[RetrievedDocument]:
        if strategy == "relevance":
            # Sort by score descending
            sorted_docs = sorted(docs, key=lambda x: x.score, reverse=True)
            # Place highest relevance at beginning and end (sandwich pattern)
            if len(sorted_docs) > 2:
                result = [sorted_docs[0]]  # Most relevant first
                middle = sorted_docs[2:]  # Remaining docs except second highest
                result.extend(middle)
                result.append(sorted_docs[1])  # Second most relevant last
                return result
            return sorted_docs
        elif strategy == "chronological":
            return sorted(
                docs, key=lambda x: x.metadata.get("date", ""), reverse=True
            )
        else:
            return docs

    def truncate_documents(
        self,
        docs: List[RetrievedDocument],
        truncation_strategy: str = "uniform",
    ) -> List[str]:
        if not docs:
            return []

        # Estimate tokens (rough heuristic: 4 chars per token)
        total_chars = sum(len(d.content) for d in docs)
        total_tokens = total_chars // 4

        if total_tokens <= self.max_context_tokens:
            return [d.format_xml() for d in docs]

        if truncation_strategy == "uniform":
            # Equal budget for each doc
            budget = (self.max_context_tokens * 4) // len(docs)
            return [d.format_xml(budget) for d in docs]

        elif truncation_strategy == "weighted":
            # Allocate by relevance score
            total_score = sum(d.score for d in docs)
            formatted = []
            for doc in docs:
                proportion = doc.score / total_score
                budget = int(self.max_context_tokens * 4 * proportion)
                formatted.append(doc.format_xml(budget))
            return formatted

        elif truncation_strategy == "head_only":
            budget = (self.max_context_tokens * 4) // len(docs)
            return [d.format_xml(budget) for d in docs]

        else:
            return [d.format_xml() for d in docs]

    def build_prompt(
        self,
        query: str,
        documents: List[RetrievedDocument],
        ordering: str = "relevance",
        truncation: str = "weighted",
    ) -> Dict[str, str]:
        # Order documents strategically
        ordered_docs = self.order_documents(documents, ordering)

        # Truncate to fit context
        formatted_sources = self.truncate_documents(ordered_docs, truncation)
        sources_text = "\n\n".join(formatted_sources)

        # Construct final prompt
        system_prompt = self.system_template.format(sources=sources_text)

        return {
            "system": system_prompt,
            "user": f"Question: {query}\n\nProvide a detailed, cited answer.",
        }

The RAGPromptBuilder class implements document ordering strategies with algorithmic complexity appropriate for production use. The order_documents method with the "relevance" strategy employs a sandwich pattern that operates in $O(n \log n)$ time due to the sorting operation, followed by $O(n)$ rearrangement. This pattern mathematically optimizes against the U-shaped attention decay curve by positioning the documents with highest relevance scores $s_1$ and $s_2$ at positions $1$ and $n$ respectively, while placing lower-relevance documents in middle positions where attention fidelity is reduced.

The truncate_documents method implements multiple strategies for handling context overflow. The "weighted" approach solves a resource allocation problem where the token budget $B$ is distributed proportionally to relevance scores. For a document $d_i$ with score $\text{score}_i$ , the allocated budget $b_i$ is:

b_i = B \times \frac{\text{score}_i}{\sum_{j=1}^{n} \text{score}_j}

where:

$b_i$ : the token budget allocated to document $i$
$B$ : the total token budget available for all retrieved content
$\text{score}_i$ : the relevance score of document $i$ , typically in $[0, 1]$
$\sum_{j=1}^{n} \text{score}_j$ : the sum of all relevance scores, used as a normalizing constant

This ensures that documents with higher relevance receive proportionally more tokens, maximizing the expected information content of the truncated context. The "uniform" strategy instead implements an egalitarian allocation where $b_i = B/n$ for all documents, which may be preferable when relevance scores are noisy or when broad coverage is more important than depth for any single document.

The build_prompt method orchestrates the full pipeline with linear complexity relative to document count: ordering documents strategically, truncating to fit context limits, and assembling the final prompt structure with proper XML formatting. The token estimation heuristic, using 4 characters per token, provides a computationally efficient approximation of true token count without requiring expensive tokenizer calls during the budgeting phase.

In[11]:

Code

# Create sample documents
docs = [
    RetrievedDocument(
        id="doc_001",
        content="The Second Circuit in Authors Guild v. HathiTrust held that creating digital copies for search and accessibility constitutes fair use. The court emphasized the transformative nature and public benefit.",
        score=0.95,
        metadata={"circuit": "2nd", "date": "2014", "type": "case_law"},
    ),
    RetrievedDocument(
        id="doc_002",
        content="NXIVM Corp. v. Ross Institute established that even commercial use can be fair if sufficiently transformative. The Second Circuit analyzed the four-factor test extensively.",
        score=0.88,
        metadata={"circuit": "2nd", "date": "2004", "type": "case_law"},
    ),
    RetrievedDocument(
        id="doc_003",
        content="The Supreme Court in Campbell v. Acuff-Rose Music established that commercial parody can qualify as fair use if transformative. This precedent influences all circuits.",
        score=0.72,
        metadata={"circuit": "SCOTUS", "date": "1994", "type": "case_law"},
    ),
    RetrievedDocument(
        id="doc_004",
        content="Recent district court decisions have expanded fair use protections for educational streaming, though these remain controversial and may be overturned.",
        score=0.45,
        metadata={"circuit": "SDNY", "date": "2023", "type": "case_law"},
    ),
]

# Build prompt with our system
builder = RAGPromptBuilder(max_context_tokens=1000)
prompt = builder.build_prompt(
    query="What constitutes fair use for educational purposes in the Second Circuit?",
    documents=docs,
    ordering="relevance",
    truncation="weighted",
)

# Prepare display strings
system_preview = (
    prompt["system"][:1500] + "..."
    if len(prompt["system"]) > 1500
    else prompt["system"]
)
user_content = prompt["user"]

Out[12]:

Console

=== SYSTEM PROMPT ===
You are a precise research assistant. Answer using ONLY the provided sources.

INSTRUCTIONS:
1. Cite every factual claim with [source_id] immediately after the claim
2. If multiple sources support a claim, cite all: [id1][id2]
3. If sources are insufficient, state: "The provided sources do not contain sufficient information."
4. When sources disagree, present all viewpoints with citations
5. Prioritize more recent sources when dates differ

AVAILABLE SOURCES:
<source id="doc_001" relevance="0.950" circuit="2nd" date="2014" type="case_law">
The Second Circuit in Authors Guild v. HathiTrust held that creating digital copies for search and accessibility constitutes fair use. The court emphasized the transformative nature and public benefit.
</source>

<source id="doc_003" relevance="0.720" circuit="SCOTUS" date="1994" type="case_law">
The Supreme Court in Campbell v. Acuff-Rose Music established that commercial parody can qualify as fair use if transformative. This precedent influences all circuits.
</source>

<source id="doc_004" relevance="0.450" circuit="SDNY" date="2023" type="case_law">
Recent district court decisions have expanded fair use protections for educational streaming, though these remain controversial and may be overturned.
</source>

<source id="doc_002" relevance="0.880" circuit="2nd" date="2004" type="case_law">
NXIVM Corp. v. Ross Institute established that even commercial use can be fair if sufficiently transformative. The Second Circuit analyzed the four-fa...

=== USER PROMPT ===
Question: What constitutes fair use for educational purposes in the Second Circuit?

Provide a detailed, cited answer.

The output shows our structured XML formatting with relevance scores, strategic ordering (highest relevance documents at boundaries), and weighted truncation that allocates more space to higher-scoring documents. Notice how doc_001 (score 0.95) appears first in the sources section and doc_002 (score 0.88) appears last, implementing the sandwich pattern. The weighted truncation gives doc_001 roughly twice as many characters as doc_004. This reflects the 2:1 ratio of their scores.