RAG Prompt Engineering: Context Placement & Citation Strategies › Worked Example: Legal Document Analysis
Language AI HandbookAll chapters RAG Prompt Engineering: Context Placement & Citation Strategies › Worked Example: Legal Document AnalysisOne section · Sections in this chapter6 / 11 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
Let's walk through a concrete example to see how all these principles come together in a realistic setting. Imagine you are building a RAG system for legal research where users ask about case law precedents.
The query: "What constitutes fair use for educational purposes in the Second Circuit?"
The retrieved documents:
- Campbell v. Acuff-Rose Music (Supreme Court, 1994), 800 tokens discussing transformative use
- NXIVM Corp. v. Ross Institute (2nd Circuit, 2004), 600 tokens on commercial vs. educational use
- Authors Guild v. HathiTrust (2nd Circuit, 2014), 1200 tokens on digital library scanning
- Cambridge University Press v. Patton (11th Circuit, 2016), 900 tokens on e-reserves (a different circuit but relevant)
Total retrieved content: 3500 tokens, which fits comfortably in most context windows. But the documents have heterogeneous authority: two are binding Second Circuit precedent (documents 2 and 3), one is persuasive Supreme Court authority (document 1), and one is persuasive authority from a different circuit (document 4). A naive prompt treats them all equally; a well-designed prompt respects this hierarchy.
Strategy A: Naive Concatenation
The following documents may be relevant:
{doc1}
{doc2}
{doc3}
{doc4}
Question: What constitutes fair use for educational purposes in the Second Circuit?
This strategy has multiple problems. There are no citation markers, so the model cannot attribute claims to specific cases. There is no indication of authority hierarchy, so the model may treat a persuasive 11th Circuit case as equally controlling to Second Circuit precedent. The most relevant documents (2nd Circuit cases) are buried in the middle, right in the lost-in-the-middle zone. And there is no instruction about what to do if the cases conflict.
Strategy B: Structured with Prioritization
You are analyzing legal precedents for fair use in the Second Circuit.
AUTHORITATIVE SOURCES (2nd Circuit precedents - prioritize these):
[Case 1] Authors Guild v. HathiTrust (2014): {first 500 tokens of doc3}
[Case 2] NXIVM Corp. v. Ross Institute (2004): {full doc2}
PERSUASIVE AUTHORITY (Other circuits):
[Case 3] Campbell v. Acuff-Rose (1994): {first 300 tokens of doc1}
[Case 4] Cambridge Univ. Press v. Patton (2016): {first 300 tokens of doc4}
INSTRUCTIONS:
- Answer based primarily on Cases 1 and 2 as controlling precedent
- Cite cases using [Case X] format
- Note circuit boundaries when citing persuasive authority
- If cases conflict, follow the most recent Second Circuit precedent
QUERY: What constitutes fair use for educational purposes in the Second Circuit?
This structure addresses each problem in Strategy A. The most relevant Second Circuit cases are placed at the beginning and end of the sources section (HathiTrust first, NXIVM last), exploiting the primacy and recency zones. Documents are labeled with meaningful identifiers that encode authority level. The instructions explicitly specify the authority hierarchy and citation format. And the fallback behavior for conflicting cases (prefer most recent Second Circuit) removes ambiguity.
The structured version is longer by roughly 100 tokens of overhead (labels, section headers, instructions), but the investment pays for itself in response quality. The model knows exactly which cases are controlling, how to cite them, and what to do when they point in different directions. The result is a response that reads like it was written by a lawyer, not a general-purpose language model.
Numerical breakdown: The total token budget is 4100 (3500 retrieved content + 600 overhead). The sandwich ordering places the 1200-token HathiTrust case (highest relevance, most recent 2nd Circuit precedent) first, which means it lands in the high-attention primacy zone. The 600-token NXIVM case (second-highest relevance, controlling 2nd Circuit precedent) goes last, landing in the recency zone. The 300-token truncated versions of documents 1 and 4 go in the middle, where their lower authority makes the attention deficit less costly. This is the sandwich pattern applied with domain knowledge: the most authoritative and most relevant documents get the premium seats.