RAG Evaluation: Metrics for Retrieval and Generation Quality › Evaluating the Retrieval Component

The retrieval stage determines the evidence available to the generator. Dense retrieval, as we covered in earlier chapters, encodes queries and documents into a shared vector space and uses similarity metrics like cosine similarity to find nearest neighbors. But geometric similarity in embedding space does not guarantee semantic relevance for the specific query at hand. A document about a related topic might be a close neighbor without containing the specific information needed. Evaluation metrics for retrieval must account for both whether relevant items appear in the result set and where they appear, since generators typically have limited context windows.

Understanding retrieval performance is foundational because it establishes an upper bound on system capability. If the retrieval component fails to fetch documents containing the answer, even the most advanced generator cannot produce a correct response. This asymmetry matters enormously in practice: you can partially compensate for an imperfect generator with better prompting or fine-tuning, but you cannot compensate for a retriever that fundamentally misses relevant information. Conversely, retrieving relevant documents is necessary but not sufficient; the ranking and presentation of those documents materially influence generation quality. We must therefore evaluate retrieval along multiple dimensions: coverage of relevant information, ranking quality, and the positioning of necessary evidence within the limited context window provided to the generator.

The evaluation of retrieval also requires making decisions about what counts as "relevant." This is more fine-grained than it first appears. A document might contain the answer to a query, which makes it highly relevant. Another might give useful background context that helps the generator reason correctly, which makes it moderately relevant. A third might mention the topic without contributing anything useful. Binary relevance judgments, which classify each document as simply relevant or not, are easier to collect but obscure these important distinctions. Graded relevance judgments offer richer information at the cost of more complex annotation guidelines and higher annotator disagreement. We will cover metrics for both approaches.

The most intuitive metrics for retrieval quality are Precision@K and Recall@K. These measure the proportion of retrieved items that are relevant, and the proportion of all relevant items that were retrieved, considering only the top $K$ results. Before diving into formulas, notice what these two metrics capture at an intuitive level. Precision@K asks: "Of the documents I handed to the generator, what fraction were useful?" Recall@K asks: "Of all the useful documents in the knowledge base, what fraction did I manage to find?"

To state this formally, let $R_K$ be the set of retrieved documents in the top $K$ positions, and $R_{rel}$ be the set of all relevant documents for a query in the entire corpus. Then:

where:

• $R_K$: the set of documents retrieved in the top $K$ positions
• $R_{rel}$: the set of all relevant documents for the query across the full corpus
• $|S|$: the cardinality (number of elements) of set $S$
• $R_K \cap R_{rel}$: the intersection, meaning documents that were both retrieved and relevant

In RAG systems, $K$ is typically small, often between 3 and 10, constrained by the generator's context window length and the cost of processing long contexts. High precision ensures the generator receives mostly useful information, while high recall ensures necessary information is not missed. Neither alone is sufficient: a system that always retrieves exactly the one most relevant document has perfect precision but poor recall for complex queries; a system that retrieves every document in the knowledge base has perfect recall but precision near zero.

The key insight is that the context window constraint makes precision especially necessary in RAG, more so than in traditional web search. When you search the web, you can scroll through results and ignore irrelevant ones. When a language model processes a context window, every irrelevant chunk occupies space that could hold necessary evidence, and research suggests that the presence of distracting information can actively degrade generation quality by confusing the model.

These metrics offer complementary perspectives on retrieval effectiveness. Precision@K focuses on the signal-to-noise ratio within the context window. This keeps the generator is not distracted by irrelevant information that might lead to hallucinations or off-topic responses. Recall@K, conversely, measures completeness, showing whether the retrieval system successfully identified all relevant sources in the knowledge base. In practice, these metrics trade off against one another: increasing $K$ typically improves recall at the expense of precision, as including more documents dilutes the concentration of relevant information.

The constraint of limited context windows fundamentally changes how we interpret these metrics compared to traditional information retrieval. In web search, you might tolerate scanning through ten results to find one useful link. In RAG, every irrelevant document actively harms performance by occupying space that could hold necessary evidence. This makes high precision needed, particularly for the earliest positions in the ranking.

Precision and Recall at K treat the retrieved set as unordered, but ranking matters in RAG. Generators tend to focus more on information appearing earlier in the context, a phenomenon known as position bias or the "lost in the middle" effect. Think of the retrieved context as a stack of papers placed in front of the generator: it reads most carefully from the top and may skim or overlook material buried deep in the pile. Ranking metrics account for where relevant items appear, rewarding systems that place the most necessary information near the top.

The position of information within the retrieved context materially impacts generation quality. Research from Liu et al. (2023) demonstrated that language models often pay less attention to content in the middle of long contexts, focusing instead on the beginning and end of the provided documents. This U-shaped attention pattern means that a highly relevant document buried at position eight contributes less to answer quality than the same document at position two. Ranking metrics capture this positional sensitivity, rewarding systems that place the most relevant evidence where the generator will utilize it most effectively.

Understanding the "lost in the middle" effect has direct practical implications for RAG system design. Some practitioners deliberately reorder retrieved documents to place the highest-scored chunks at the very beginning and end of the context window, with less necessary supporting documents in the middle. This "placement strategy" exploits the known attention patterns of language models and can improve answer quality without any changes to the retrieval or generation components themselves. Evaluating systems with ranking-aware metrics ensures you are building this kind of positional awareness into your optimization process.

Not all relevant documents are equally relevant. A chunk that directly answers the query is more useful than one that gives tangential background. A document containing a precise statistic is more useful than one that vaguely mentions the same topic. Binary relevance judgments force evaluators to collapse these real distinctions into a single bit, losing information that matters for system quality.

Binary relevance judgments simplify evaluation but obscure important distinctions in information quality. A document containing the exact answer to a query is fundamentally more useful than one that merely mentions related concepts. Think of relevance as a spectrum rather than a switch: at one end, a document that directly and completely answers the query; at the other end, a document that is completely off-topic; and in between, varying degrees of partial relevance, topical relevance, and contextual usefulness. Graded relevance acknowledges these differences, letting evaluators to distinguish between perfect matches, highly relevant documents, marginally relevant background material, and irrelevant content. This richer annotation lets metrics that better reflect the actual utility of retrieved documents.

Normalized Discounted Cumulative Gain (NDCG) handles graded relevance scores (e.g., 0 to 3) rather than binary judgments. The construction of NDCG proceeds in three steps, each building on the previous.

Step 1: Cumulative Gain (CG) at position $K$ simply sums the graded relevance scores of retrieved documents, ignoring their positions:

where:

• $\text{rel}_i$: the graded relevance score (e.g., 0 to 3) of the document at position $i$
• $K$: the number of top documents considered in the ranking

CG has an obvious flaw: it assigns equal value to relevant documents regardless of where they appear. A system that returns the most relevant document at position 5 scores identically to one that returns it at position 1. Since position matters for RAG, we need to discount contributions from lower positions.

Step 2: Discounted Cumulative Gain (DCG) penalizes relevant items that appear lower in the ranking using a logarithmic discount factor:

where:

• $\text{rel}_i$: the graded relevance score of the document at rank $i$
• $i$: the position of the document in the ranked list (starting from 1)
• $2^{\text{rel}_i} - 1$: the exponential gain function, giving exponentially more weight to higher relevance grades
• $\log_2(i + 1)$: the logarithmic discount factor that reduces the contribution of documents at lower ranks

The logarithm base ensures that each position contributes less than the previous, but the difference between positions 1 and 2 is larger than between positions 9 and 10. This shows the reality that early positions receive disproportionate attention. We use $2^{\text{rel}} - 1$ to give exponentially more weight to highly relevant documents. With a 0-3 grading scale, the gains are 0, 1, 3, and 7 for grades 0, 1, 2, and 3 respectively. This means a grade-3 document contributes seven times as much gain as a grade-1 document, not just three times as much, which shows the intuition that a perfect answer is substantially more useful than a marginally useful one.

The DCG formula elegantly combines two important principles: the value of relevance and the cost of position. The exponential term in the numerator ensures that finding a highly relevant document (score 3) is substantially more useful than finding a moderately relevant one (score 2), specifically about $7/3 \approx 2.3$ times as useful based on gain values. The logarithmic denominator implements a smooth discounting of lower positions. This shows the diminishing attention that both human users and language models pay to later items in a list.

Step 3: Normalized DCG (NDCG) divides by the Ideal DCG (IDCG), which is the DCG of the perfect ranking where all relevant documents appear at the top in order of decreasing relevance:

where:

• $\text{DCG@K}$: the Discounted Cumulative Gain at position $K$ for the actual ranking
• $\text{IDCG@K}$: the Ideal DCG at position $K$, representing the maximum possible DCG if documents were ranked in perfect order of relevance

NDCG ranges from 0 to 1, where 1 is a perfect ranking. The reason is that the normalization by IDCG lets fair comparison across queries of varying difficulty. A query with many highly relevant documents in the corpus will have a high IDCG, while a query with only one relevant document will have a low IDCG. Without normalization, the former would dominate aggregate metrics regardless of retrieval quality. NDCG solves this by measuring the achieved fraction of the theoretically possible gain. This gives a standardized score that shows both the quality and ordering of retrieved documents.

In RAG evaluation, graded relevance is important because context windows are limited: placing a highly relevant chunk at position 8 when it should be at position 1 means it might get truncated or receive less attention from the generator. NDCG captures this cost precisely, which makes it the preferred metric when you have access to graded relevance judgments.

With Precision@K, Recall@K, MRR, MAP, and NDCG available, the natural question is which to use when. The answer depends on your query types and evaluation goals. For simple factoid questions where a single relevant document suffices, MRR captures the most important information: how quickly does the system return a useful answer? For complex analytical questions requiring multiple sources, MAP or NDCG better reflect whether the system found and ranked all relevant evidence.

If your knowledge base contains documents with varying degrees of relevance (not just relevant or not), NDCG is superior because it uses the full relevance signal. If you only have binary relevance judgments, Precision@K, Recall@K, MRR, and MAP are your options. In practice, many production RAG evaluations report Precision@K and Recall@K alongside MRR or NDCG, giving a complete picture of both coverage and ranking quality.

One practical consideration is the cost of creating relevance judgments. Binary judgments (relevant or not) are faster and cheaper to annotate than graded judgments (0-3). For initial evaluation pipelines, binary judgments with Precision@K and MRR often give enough signal to guide development. Graded judgments and NDCG become more useful once you need to distinguish between systems that already reach decent binary precision and recall but differ in the quality of their ranked order.