Dense Retrieval: Semantic Search & Bi-Encoder Implementation

In the previous chapter, we examined the RAG architecture and how it combines retrieval with generation. At the heart of any RAG system lies a necessary question: how do we find the most relevant documents for a given query? The answer to that question determines whether the LLM receives the right context to produce an accurate, grounded response, or whether it receives noise that misleads it. Traditional search engines rely on lexical matching, counting how often query terms appear in documents and scoring results by term frequency statistics. This approach has served us well for decades and remains the backbone of most search infrastructure today.

But lexical matching has a basic blind spot: it operates entirely on surface form. If the document uses different words to express the same concept, a lexical retriever will miss it entirely. What happens when a patient asks "heart attack symptoms" but the medical textbook discusses "myocardial infarction warning signs"? What happens when a software engineer searches for "how to speed up my code" but the documentation describes "performance optimization techniques"? The words are completely different, yet the meaning is identical. A retriever that counts term overlap would score these relevant documents at zero, returning results that match the words but not the intent.

Dense retrieval addresses this basic limitation by representing both queries and documents as continuous vectors in a shared semantic space. Rather than matching exact words, dense retrieval measures the similarity between the meanings of queries and documents. A query about "climate change impacts" can match documents discussing "global warming effects" because both map to nearby points in the embedding space, even though they share no common terms. Think of this as a map of meaning: semantically similar concepts are placed close together, and retrieval becomes a nearest-neighbor search in that space.

This shift from discrete term matching to continuous semantic similarity is one of the most significant advances in information retrieval in the past decade. It moves retrieval from pattern matching toward language understanding. Building on the transformer architectures and embedding techniques we have explored throughout this book, dense retrieval enables systems that compare meaning rather than merely counting words. The bi-encoder architecture at the heart of dense retrieval allows us to encode entire corpora offline, then retrieve relevant passages at query time with a single vector similarity computation.

Understanding dense retrieval deeply matters because it forms the foundation of modern RAG pipelines, semantic search products, question answering systems, and knowledge-intensive language tasks. Every time you receive an accurate, knowledge-grounded response from a production LLM assistant, there is almost certainly a dense retrieval system working behind the scenes. In this chapter, we build that system from the ground up: starting from the vocabulary mismatch problem that motivates it, through the bi-encoder architecture that makes it practical, into the similarity metrics and training objectives that make it accurate, and finally through a complete implementation you can adapt for your own applications.

Recall from Part II that BM25 retrieves documents by computing term-frequency statistics: documents score higher when they contain more query terms, with diminishing returns for repeated terms and penalties for common words. This approach works remarkably well for navigational queries, entity lookups, and any search where the vocabulary of the query and the relevant documents naturally aligns. The design is elegant in its simplicity: store an inverted index mapping each term to the documents that contain it, and at query time, look up each query term and aggregate the scores.

The vocabulary mismatch problem becomes acute in several important scenarios that arise constantly in real applications. Consider searching a medical knowledge base with the query "heart attack symptoms." A relevant document might discuss "myocardial infarction warning signs" without ever using the words "heart" or "attack." BM25 would score this document at zero because it shares no terms with the query. Yet a physician would immediately recognize these as describing the same condition. The mismatch here is between everyday language and technical terminology, and it is systematic: patients speak in plain language while medical literature uses precise clinical terms.

The vocabulary gap manifests in many forms, and each form defeats lexical retrieval in a different way:

• Synonyms and paraphrases: "car" vs "automobile," "purchase" vs "buy," "use" vs "utilize." Natural language has extraordinary breadth of expression for any given concept.
• Technical versus casual language: "myocardial infarction" vs "heart attack," "hypertension" vs "high blood pressure." Users often prefer informal phrasing while documents use formal or specialized terms.
• Abbreviations and expansions: "ML" vs "machine learning," "NLP" vs "natural language processing." Abbreviations are common in casual queries but documents may spell things out.
• Conceptual similarity without lexical overlap: "renewable energy policy" might relate to documents about "solar panel subsidies" or "wind farm regulation," even though these share no query terms.
• Different grammatical forms and derivations: "optimize" vs "optimization," "predict" vs "prediction." Stemming helps partially, but doesn't solve cross-form semantic relationships.

Dense retrieval sidesteps vocabulary mismatch entirely by operating in a different representational space. Instead of asking "which words match?", it asks "which meanings match?" By encoding both queries and documents into a continuous vector space where semantically similar texts cluster together, dense retrieval can find relevant documents regardless of the specific words they use.

The key insight is that transformer models, pre-trained on enormous amounts of text, develop internal representations that capture meaning at a level far beyond surface form. The model has seen "myocardial infarction" and "heart attack" used in similar contexts thousands of times, so their representations end up in similar regions of the embedding space. Dense retrieval uses this learned semantic geometry. When we ask the model to encode a query, it produces a vector that points in a direction in semantic space that represents the query's meaning. Document vectors that point in similar directions represent documents about related concepts.

This is a fundamentally different philosophy from lexical retrieval. In lexical retrieval, the representation is determined entirely by the words: the same words always produce the same representation, regardless of context. In dense retrieval, the representation is learned from data and captures distributional semantic relationships. Words that appear in similar contexts across millions of documents end up with similar representations, creating an implicit semantic knowledge graph embedded in the geometry of the vector space.

The dominant architecture for dense retrieval is the bi-encoder, which uses two separate encoder networks to produce embeddings for queries and documents independently. This separation is important for efficiency at scale, and understanding why requires us to think carefully about what happens during retrieval.

Think of a bi-encoder as two photographers: the query encoder takes a photograph of the question, and the document encoder takes a photograph of each document. The photographs are all printed in the same format, so you can compare them side by side using a simple similarity measure. Because the document photographs can be taken before any query arrives (offline), you only need to take the query photograph at search time and then compare it against the pre-existing document photographs. This is what makes the bi-encoder scalable: the expensive work of encoding documents happens once, not once per query.

The bi-encoder consists of two components that work in tandem to transform text into comparable numerical representations. The first component is the query encoder, denoted $E_q$, which takes a natural language query $q$ as input and produces a dense vector representation. The second component is the document encoder, denoted $E_d$, which performs the analogous transformation for documents.

Formally, these transformations are:

• Query encoder $E_q$: Maps a query $q$ to a dense vector $\mathbf{q} = E_q(q) \in \mathbb{R}^D$
• Document encoder $E_d$: Maps a document $d$ to a dense vector $\mathbf{d} = E_d(d) \in \mathbb{R}^D$

Both encoders are typically initialized from the same pre-trained transformer (such as BERT, as we covered in Part XXIV), and they may share weights or be fine-tuned independently. The encoders produce embeddings of the same dimensionality $D$, letting direct comparison. This shared dimensionality is needed because it allows us to measure distances and angles between query and document vectors in the same geometric space. Without a shared dimensionality, there would be no principled way to compare a query vector to a document vector.

In practice, the query and document encoders are often initialized identically and may even share weights during training. Weight sharing reduces the total parameter count and can improve training stability. Some architectures allow the query and document encoders to diverge during fine-tuning, which can be beneficial when the distributional characteristics of queries and documents are very different. Short, keyword-like queries and long, paragraph-length documents are stylistically distinct, and separate encoders give the model flexibility to specialize. For many practical applications, however, a single shared encoder works well and is simpler to maintain.

A key challenge in building these encoders is converting the variable-length output of a transformer into a single fixed-size vector suitable for comparison. Recall from our BERT chapters that when a BERT model processes an input sequence, it produces a hidden state vector for each token in the sequence. A sentence of 20 words produces 20 hidden state vectors (plus special tokens), while a paragraph of 200 words produces 200 hidden state vectors. We need to compress this variable-length sequence into a single fixed-size vector, a process called pooling.

BERT-based encoders typically extract the representation of the special [CLS] token from the final layer:

where:

• $\mathbf{q}$ is the dense vector representation of the query
• $\text{BERT}(q)$ is the sequence of hidden states from the last layer of the BERT model
• $[\text{CLS}]$ indicates extraction of the vector corresponding to the special classification token, which is designed to aggregate sequence-level information during pre-training

The [CLS] token was specifically designed during BERT's pre-training to aggregate information from the entire sequence. During the next-sentence prediction pre-training task, the [CLS] representation is used to classify whether two sentences are consecutive, forcing it to encode sequence-level rather than token-level information. This makes it a natural candidate for creating a single vector that summarizes the entire input.

Alternatively, to capture information distributed across the entire sequence rather than relying on a single token, some models compute the average of all token representations. This approach, known as mean pooling, treats each token's contribution equally:

where:

• $\mathbf{q}$ is the dense vector representation of the query
• $n$ is the number of tokens in the query
• $\text{BERT}(q)_i$ is the vector representation of the $i$-th token
• $\sum$ computes the element-wise sum of all token vectors, and dividing by $n$ finds the geometric centroid

The intuition behind mean pooling is that important semantic information may be spread across multiple tokens rather than concentrated in the [CLS] token. By averaging, we create a representation that balances contributions from all parts of the input. Consider a sentence like "The quick brown fox jumped over the lazy dog." The [CLS] representation tries to summarize this whole sentence, but mean pooling creates a vector that represents the average meaning of all the individual tokens. For retrieval purposes, mean pooling often works slightly better because it ensures no important content is lost.

In practice, the choice between [CLS] pooling and mean pooling affects retrieval quality, and different models adopt different strategies based on their training objectives. Empirically, models trained with mean pooling as part of their objective tend to perform better when evaluated using mean pooling, and likewise for [CLS] pooling. The Sentence-BERT family of models (which we use in our implementation) were specifically trained with mean pooling and perform substantially better with it.

The bi-encoder's separation of query and document encoding enables a necessary optimization: pre-computation of document embeddings. To appreciate why this matters, consider the computational demands of a retrieval system at real scale.

In a production retrieval system serving a corpus of 10 million documents, we need to compare every query against every potential document to find the most relevant ones. If we process each query-document pair jointly, a single query requires 10 million transformer forward passes. A transformer forward pass on a modern BERT-sized model takes approximately 10 milliseconds on a GPU. This means a single query would take 100,000 seconds, or about 27 hours. That is obviously not a retrieval system; it is a computational nightmare.

With the bi-encoder, we pre-compute document embeddings once. At query time, we run the transformer once for the query (taking 10 milliseconds), then compute dot products between the query vector and all 10 million pre-computed document vectors. Computing 10 million dot products in a 768-dimensional space takes microseconds using vectorized operations. The total query latency drops from 27 hours to roughly 50 milliseconds. This is the bi-encoder's core value proposition: it separates the expensive transformer computation (done once per document, offline) from the cheap similarity computation (done at query time, over pre-computed vectors).

This contrasts with cross-encoders, which concatenate the query and document and process them jointly through a single transformer. Cross-encoders can capture fine-grained interactions between query and document tokens. The attention mechanism allows every query token to attend to every document token, letting the model to reason about how specific parts of the document relate to specific parts of the query. This richer interaction produces more accurate relevance judgments in controlled experiments.

However, cross-encoders require running the transformer once for every query-document pair. For a corpus of 10 million documents, this means 10 million transformer forward passes per query, which is computationally infeasible for first-stage retrieval. Cross-encoders therefore serve a different role in the retrieval pipeline: they are used as rerankers that refine the top candidates returned by a faster first-stage retriever. We will explore reranking with cross-encoders in a later chapter.

The bi-encoder architecture trades some modeling power for massive efficiency gains:

We will explore reranking with cross-encoders in a later chapter, where they serve as a second-stage refinement over bi-encoder candidates. The combination of bi-encoder retrieval followed by cross-encoder reranking is a powerful and widely used two-stage pipeline.

Beyond [CLS] and mean pooling, several other pooling strategies have been explored. Max pooling takes the maximum value in each dimension across all token vectors, which can emphasize strong signals from individual tokens. Weighted mean pooling assigns importance weights to tokens before averaging, potentially downweighting stopwords and upweighting content words. Some specialized architectures produce multiple vectors per document (one per sentence or paragraph) to support more granular retrieval.

The Sentence Transformers library, which we use in our code implementation, uses mean pooling combined with optional L2 normalization by default. This combination has proven reliable across many retrieval benchmarks. The normalization step ensures that embedding magnitudes are uniform, which simplifies the similarity computation and prevents magnitude biases from distorting retrieval results.

One important consideration is the maximum sequence length. Transformer models have a fixed context window, typically 512 tokens for BERT-based models. Documents longer than this limit must be truncated or split into multiple chunks. In RAG systems, documents are usually pre-chunked into passages of a few hundred words specifically to fit within these limits. Retrieval quality depends on both chunk size and overlap, including whether chunks respect sentence boundaries. Longer chunks preserve more context but may dilute relevance signals; shorter chunks are more focused but may miss context needed to answer the query.

Once we have query and document embeddings, we need a similarity function to rank documents. The three most common metrics are dot product, cosine similarity, and Euclidean distance. Each of these metrics captures a different notion of what it means for two vectors to be "similar," and understanding their geometric interpretations helps us choose the right metric for a given application.

Think of the embedding space as a high-dimensional room where every piece of text has a specific location. Similarity metrics are different ways to measure how "close" two locations are. The dot product measures closeness while also caring about how far each vector is from the origin. Cosine similarity measures only the angle between vectors, ignoring how far they are from the origin. Euclidean distance measures the straight-line distance between the two locations in the room. Each notion of closeness has its uses, and the right choice depends on how the embeddings were trained and what properties you want retrieval to have.

The dot product quantifies the similarity between two vectors by aggregating their aligned components. To understand this intuitively, imagine two vectors as arrows pointing in some direction in high-dimensional space. The dot product measures how much one vector "projects" onto another, combining information about both their directions and their lengths. For vectors $\mathbf{q}$ and $\mathbf{d}$, the dot product is defined as:

where:

• $\text{sim}(\mathbf{q}, \mathbf{d})$ is the similarity score
• $\mathbf{q}, \mathbf{d}$ are the dense vectors for the query and document
• $D$ is the dimensionality of the embedding space (e.g., 768)
• $q_i, d_i$ are the $i$-th scalar components of each vector
• $\sum$ aggregates alignment across all $D$ dimensions

The geometric interpretation of this formula is instructive. Each dimension of the embedding space captures some aspect of meaning. When both $q_i$ and $d_i$ are large and positive, their product contributes positively to the similarity, showing that both the query and document exhibit that particular semantic feature strongly. Conversely, when one component is positive and the other is negative, their product is negative, reducing the overall similarity score. The dot product is therefore highest when the two vectors point in the same direction and have large magnitudes.

The dot product is computationally efficient and captures both the alignment of vectors (their angular similarity) and their magnitudes. Larger embeddings produce larger dot products, which can be useful when magnitude carries semantic meaning. For example, longer, more detailed documents might have larger embedding norms, and this property could be desirable if we want to favor complete, information-rich documents over brief ones.

Many dense retrieval models, including the original DPR (Dense Passage Retrieval), use dot product similarity because it can be computed extremely efficiently using matrix multiplication. Given a query vector and a matrix of document vectors, all similarity scores can be computed in a single matrix-vector multiplication using highly optimized BLAS libraries that run at near-peak hardware throughput.

While the dot product captures both direction and magnitude, there are situations where we want to focus purely on semantic direction, ignoring how "long" the vectors are. Cosine similarity addresses this need by normalizing the dot product by the magnitudes of both vectors, measuring only the angular alignment:

where:

• $\cos(\mathbf{q}, \mathbf{d})$ is the cosine similarity score, ranging from $-1$ to $+1$
• $\mathbf{q} \cdot \mathbf{d}$ is the unnormalized dot product (raw alignment)
• $\|\mathbf{q}\|, \|\mathbf{d}\|$ are the Euclidean norms (lengths) of the vectors
• $\sqrt{\sum q_i^2}$ computes the vector's magnitude as the square root of the sum of squared components

The cosine similarity ranges from $-1$ (vectors pointing in opposite directions) to $+1$ (vectors pointing in the same direction), with $0$ showing orthogonality. By removing magnitude effects, cosine similarity focuses purely on semantic direction. This is particularly useful when we want to compare texts of different lengths on an equal footing, since a longer document might naturally produce a larger-magnitude embedding without being more relevant to any particular query.

The key insight is that by normalizing, we make the similarity score a function only of the angle between the vectors. Two documents that say exactly the same thing but one is much longer (perhaps with more examples and explanations) should have similar cosine similarity to a query, even if the longer document's raw embedding vector is larger in magnitude. This length-invariant behavior is often what we want in retrieval.

A useful equivalence emerges when embeddings are L2-normalized, meaning each vector is divided by its magnitude so that $\|\mathbf{q}\| = \|\mathbf{d}\| = 1$. In this case, cosine similarity equals the dot product:

Here, $\hat{\mathbf{q}}$ and $\hat{\mathbf{d}}$ are the unit-length vectors derived by dividing $\mathbf{q}$ and $\mathbf{d}$ by their respective norms. The dot product operation, applied to these normalized vectors, directly yields the cosine of the angle between the original vectors.

This equivalence is practically very useful: we can pre-normalize all document embeddings once during indexing, then use the faster dot product operation at query time while still computing cosine similarity. This is exactly what many production systems do, including most models in the Sentence Transformers library. Pre-normalizing once means we never need to compute norms at query time, keeping retrieval latency minimal.

The third common metric takes a different perspective entirely. Rather than measuring how vectors align, Euclidean (L2) distance measures the straight-line distance between vectors in the embedding space:

where:

• $\text{dist}(\mathbf{q}, \mathbf{d})$ is the Euclidean distance (lower means more similar)
• $\mathbf{q} - \mathbf{d}$ is the difference vector, computed element-wise
• $(q_i - d_i)^2$ is the squared difference in dimension $i$, which is always non-negative
• $\sqrt{\cdot}$ converts the sum of squared differences to linear distance units

Unlike the similarity metrics above, smaller Euclidean distances indicate greater similarity. This is an important distinction to keep in mind when implementing retrieval systems: with Euclidean distance, we search for the nearest neighbors rather than the highest-scoring documents.

Euclidean distance is related to dot product for normalized vectors. When vectors have unit length, minimizing Euclidean distance is mathematically equivalent to maximizing the dot product. To see why, expand the distance formula:

Since $2$ and $1$ are constants, minimizing $\|\mathbf{q} - \mathbf{d}\|^2$ is equivalent to maximizing $\mathbf{q} \cdot \mathbf{d}$, which for normalized vectors equals cosine similarity. This relationship allows certain approximate nearest neighbor indexing algorithms designed for Euclidean distance to be adapted for dot product or cosine similarity retrieval, which is important for efficient large-scale search.

The choice of similarity metric should match the training objective used when the embedding model was created. This is not a trivial constraint: using the wrong metric can substantially degrade retrieval quality even for an otherwise excellent model.

• Models trained with dot product loss perform best with dot product similarity at inference time
• Models trained with cosine loss perform best with cosine similarity
• Many embedding models normalize their outputs, making the choice less necessary in practice

In practice, cosine similarity (or equivalently, dot product with normalized embeddings) is the most common choice because it is reliable to variations in embedding magnitude and provides easily interpretable scores. When similarity ranges from $-1$ to $+1$, it becomes straightforward to set thresholds and compare scores across different queries and different corpora. A score of 0.8 means something consistent regardless of whether the query is long or short, whether the document is a sentence or a paragraph.

One subtle consideration is how the model was trained with respect to embedding norms. Some contrastive learning setups allow the model to encode confidence or importance in the embedding norm, making dot product the preferred metric. Others explicitly normalize embeddings during training, making cosine similarity the correct choice. Always check the documentation for any pre-trained embedding model to understand what metric it expects.

Understanding when to use dense versus sparse retrieval requires examining their complementary strengths and weaknesses in detail. These are not competing technologies where one is simply better; they are different tools suited to different problems, and the best production systems often combine both.

Think of sparse retrieval as a librarian who finds books by searching the catalog for exact title and keyword matches, and dense retrieval as a librarian who understands the topic you are researching and can recommend relevant books even when they use completely different terminology. The first librarian works quickly and predictably without making creative associations. The second understands your intent but might occasionally recommend something that sounds related but misses the point.

Dense retrieval excels in scenarios where semantic understanding matters more than exact term matching:

Semantic matching: Dense retrieval captures meaning beyond surface forms. The query "best laptop for programming" matches documents about "developer-friendly notebooks with good keyboards" even without shared terms. This is the core value proposition: meaning-based matching rather than word-based matching.

Handling synonyms and paraphrases: Medical queries like "hypertension treatment" naturally match documents about "high blood pressure medication" because both concepts map to similar regions in the embedding space. The model has learned from large training data that these terms are used interchangeably in similar contexts.

Cross-lingual retrieval: With multilingual encoders trained on parallel corpora, the same query can retrieve relevant documents in different languages, as semantically equivalent text in different languages clusters together in the embedding space. This is particularly valuable for organizations with multilingual knowledge bases.

Tolerance of typos and minor variations: Minor spelling variations like "recieve" vs "receive" often produce similar embeddings because the transformer tokenizer breaks them into similar subword units, and the model has learned to handle such variations. Exact-match systems would fail entirely on misspelled queries.

Natural language and conversational queries: Queries that read like natural sentences ("What are the side effects of ibuprofen?") are well-handled by dense retrieval because the model understands intent rather than relying on specific keyword matches. This is especially important as voice search and conversational interfaces become more common.

Sparse retrieval (BM25, TF-IDF) maintains important advantages in several areas that should not be dismissed:

Exact match requirements: When you search for specific identifiers like "CVE-2024-1234," "iPhone 15 Pro Max," or "Python 3.11.2," exact term matching is needed. Dense models may confuse similar but distinct identifiers, potentially returning results for "CVE-2024-1235" or "iPhone 15 Pro" when you need an exact match. This distinction can be necessary in security and legal work as well as technical documentation.

Rare terms and proper nouns: Uncommon terms carry high information value in BM25 through high IDF (inverse document frequency) weights. Dense models may underweight rare terms that were not well-represented in training data, potentially missing specialized topics. A query about an obscure historical figure or a niche technical protocol may be better served by a system that knows rare term occurrences are highly informative.

Interpretability and debugging: BM25 scores are explainable. You can tell a user: "This document ranked high because it contains the query terms 'machine' (3 times) and 'learning' (5 times)." Dense similarity scores are opaque black boxes. When retrieval goes wrong, debugging a BM25 system is straightforward; debugging a dense retrieval system requires understanding what the model learned and why it assigned certain similarity scores.

Zero-shot generalization: BM25 works immediately on any corpus without training or fine-tuning. Dense retrievers need training data that matches the target domain. For organizations with specialized text (legal contracts, scientific literature, source code, internal documentation), building training data and fine-tuning models requires significant investment.

Memory efficiency: BM25 uses inverted indices that scale efficiently to billions of documents with compact storage. Dense retrieval requires storing one full embedding vector per document, which at 768 dimensions in float32 requires approximately 3 KB per document. For a corpus of 100 million documents, that is 300 GB of embedding storage before any indexing overhead.

Dense retrieval struggles predictably with:

• Keyword-heavy queries: Searching for "Python pandas DataFrame merge" requires exact library and function names. The user knows the specific API they need and does not want semantically related but incorrect functions.
• Out-of-domain text: Models trained on Wikipedia may perform poorly on legal contracts, clinical notes, or source code because the semantic relationships in those domains differ substantially from general text.
• Negation: "hotels without pools" might match documents about "hotels with pools" because both contain similar concepts. Dense models struggle with negation because their training rarely includes negative contrastive examples at this semantic level.
• Numerical and logical constraints: "apartments under $2000/month in Seattle" requires understanding numerical constraints and geographic specificity, not just semantic similarity.

Sparse retrieval struggles predictably with:

• Vocabulary mismatch: "affordable housing" vs "low-cost apartments" versus "subsidized residence" all mean similar things but share no terms.
• Paraphrased queries: "what causes climate change" vs "global warming factors" uses different vocabulary for the same question.
• Conceptual queries: "books like Harry Potter" requires understanding the genre and writing style, including narrative characteristics, rather than just term overlap.
• Short queries: Single-word or two-word queries provide little context for term weighting. "Python" could mean the programming language, the snake, or the British comedy group.

Given these complementary strengths, many production systems combine both approaches. A typical hybrid retrieval strategy runs BM25 and dense retrieval in parallel, creating two ranked lists of candidates, then merges them using reciprocal rank fusion or learned score combination. The merged list captures both lexical precision and semantic recall, performing better than either approach alone on most benchmarks.

The simplest combination method is Reciprocal Rank Fusion (RRF), which scores each document by the sum of the reciprocals of its ranks in each list. If a document appears at rank 3 in the BM25 list and rank 7 in the dense list, its RRF score is $\frac{1}{3+60} + \frac{1}{7+60}$ where 60 is a smoothing constant. Documents that appear highly ranked in both lists receive the highest combined scores, while documents unique to one list receive moderate scores based on their rank in that list.

We will explore hybrid search techniques in detail in a later chapter on combining retrieval signals, including how to learn optimal combination weights from data and how to handle score normalization across different retrieval systems.

Creating effective dense retrievers requires specialized training to produce embeddings where similar queries and documents cluster together. This section covers the key components of the training process, from the mathematical objective that guides learning to the practical considerations of data collection and negative sampling. Understanding training is important for practitioners who fine-tune their own models and for anyone who wants to understand why off-the-shelf models have certain strengths and weaknesses.

The basic challenge in training a dense retriever is supervision: how do we tell the model what "similar" means? Unlike classification tasks where labels are clear-cut, relevance in retrieval is gradational and context-dependent. A document about photosynthesis is highly relevant to "how do plants make food" but somewhat relevant to "renewable energy sources." The training framework needs to capture this nuance.

The goal of dense retriever training is to learn an embedding space where relevant query-document pairs have high similarity and irrelevant query-document pairs have low similarity. The key insight is that we can formulate this as a contrastive learning problem: teach the model to distinguish between documents that satisfy the information need and documents that do not.

Given a query $q$, a relevant (positive) document $d^+$, and irrelevant (negative) documents $\{d^-_1, d^-_2, ..., d^-_k\}$, we want:

where:

• $\text{sim}(\cdot)$ is the similarity function (e.g., dot product or cosine)
• $E_q(q), E_d(\cdot)$ are the encoder embeddings for queries and documents respectively
• $d^+$ is the relevant (positive) document
• $d^-_i$ is the $i$-th irrelevant (negative) document
• $\forall i$ indicates the condition should hold for all negative samples

The intuition behind this formulation is geometric: we want the query embedding to lie closer to the positive document embedding than to any negative document embedding in the vector space. The training loss operationalizes this geometric constraint.

The most common loss function is the contrastive loss (also called InfoNCE loss or NT-Xent loss). This function treats retrieval as a multi-class classification task over the set of candidates in a training batch. It first computes the probability that the positive document is the correct match among all candidates (positive plus negatives), then minimizes the negative log of that probability.

To build the loss step by step: we start with the query embedding $\mathbf{q}$ and compute its similarity to the positive document embedding $\mathbf{d}^+$ and each negative embedding $\mathbf{d}^-_i$. We then convert these similarities to a probability distribution using softmax with temperature $\tau$:

where:

• $P(d^+|q)$ is the probability that $d^+$ is the correct match for query $q$
• $\mathcal{L}$ is the contrastive loss we minimize during training
• $\mathbf{q}, \mathbf{d}^+, \mathbf{d}^-_i$ are the embeddings for query, positive, and negative documents
• $\tau$ is the temperature parameter that controls the sharpness of the probability distribution
• $k$ is the number of negative samples per positive

The numerator $\exp(\text{sim}(\mathbf{q}, \mathbf{d}^+) / \tau)$ is the exponentiated similarity score for the positive pair. The denominator normalizes this by summing the exponentiated scores for the positive document and all $k$ negative documents. The resulting $P(d^+|q)$ is the probability that the model assigns to the positive document being the correct match, given all the candidates.

The loss $\mathcal{L} = -\log P(d^+|q)$ is minimized when $P(d^+|q)$ approaches 1, which happens when the positive document's similarity is much higher than all negative documents' similarities. Maximizing this probability is exactly what we want: make the positive pair the most similar, by a large margin.

The temperature parameter $\tau$ deserves special attention. When $\tau$ is small (close to 0), the exponential function amplifies differences between similarity scores. This makes the model more confident in its distinctions and giving a sharper training signal. When $\tau$ is large, the probability distribution becomes more uniform. This provides a weaker gradient signal. Finding the right temperature, typically in the range of 0.05 to 0.1 for dense retrieval, is often done through experimentation, though some training frameworks learn it automatically.

Dense retrievers require training data consisting of query-document pairs with relevance labels. The quality and diversity of training data is one of the most important factors determining retrieval performance. Common sources include:

Natural Questions (NQ): Google's dataset of real questions from Google Search users, paired with Wikipedia passages containing answers. This provides a natural distribution of factual queries and their supporting evidence. The queries reflect how real users phrase questions, which is valuable for training a retriever that handles diverse natural language inputs. The limitation is that coverage is restricted to knowledge available on Wikipedia.

MS MARCO: Microsoft's large-scale reading comprehension dataset with approximately 500,000 queries drawn from Bing search logs, each paired with relevant passages from web documents. Its scale and web domain make it the most popular dataset for training general-purpose retrievers. Models trained on MS MARCO generalize reasonably well to many retrieval tasks, which makes it a common pre-training choice before domain-specific fine-tuning.

Synthetic data from LLMs: A recent and increasingly popular approach uses large language models to generate queries for existing documents. Given a passage, an LLM generates realistic questions that the passage would answer. This allows creating training data for any domain without requiring expensive human annotation. The quality of synthetic data has improved substantially as LLMs have become more capable, and recent work shows that models trained on synthetic data can approach the performance of models trained on human-annotated data.

Click logs from production systems: In deployed search or recommendation systems, user clicks provide implicit relevance signals. Documents that users click after submitting a query are treated as positive examples. This provides enormous amounts of training signal at low cost. This approach has been used by large-scale systems like Google, Bing, and Amazon to continuously improve their retrieval models. The challenge is handling noise: users sometimes click irrelevant results out of curiosity or mistake.

Domain-specific curated datasets: For specialized domains like legal, medical, or scientific retrieval, researchers and organizations have created curated datasets. Examples include BEIR (a benchmark with 18 diverse retrieval datasets), BioASQ for biomedical question answering, and MLDR for multilingual dense retrieval. Fine-tuning on these domain-specific datasets is often necessary for high-quality retrieval in specialized applications.

The choice of negative documents significantly impacts training quality, and this is one of the most studied aspects of dense retrieval training. The intuition is simple: the negatives define what "not relevant" means, and the model learns to distinguish positives from negatives. If negatives are too easy (obviously irrelevant), the model learns weak distinctions that do not help at inference time. If negatives are too hard (indistinguishable from positives), the training signal becomes noisy and inconsistent.

Random negatives: Sample random documents from the corpus. This is simple but often too easy. Random documents from a large corpus are typically obviously irrelevant to any given query. A retriever trained only on random negatives learns to distinguish relevant documents from random text, which is easy and does not prepare it for the harder cases where it needs to distinguish between two semantically similar documents, only one of which answers the query.

BM25 negatives: Use BM25 to find documents that contain query terms but are not relevant. These are called "hard negatives" because they share surface features with the query but differ semantically, forcing the model to learn representations that go beyond lexical overlap. For example, for the query "causes of inflation," a BM25 negative might be an article that mentions "inflation" and "causes" frequently but discusses historical inflation rather than economic causes. Training on such negatives teaches the model to distinguish semantic nuances.

In-batch negatives: Use positive documents from other queries in the same training batch as negatives for the current query. This approach is computationally efficient because it requires no additional encoding: the positive document embeddings already computed for other queries can be reused as negatives for the current query. With a batch size of 64, each query has 63 in-batch negatives at no extra encoding cost. The main limitation is that in-batch negatives may be easy (the other queries in the batch are typically about different topics), but with large batch sizes and diverse training data, in-batch negatives can be quite effective.

Self-mined hard negatives: Use the current model to find high-scoring documents that are not labeled as relevant. These are the hardest possible negatives because the current model finds them confusable with true positives. After each training epoch (or after a fixed number of steps), re-rank the corpus using the current model and mine new hard negatives. This iterative mining process progressively makes training harder, keeping the model in a regime where it is always learning something challenging.

Research has consistently shown that combining strategies works best. A typical approach: start training with in-batch negatives plus some BM25 negatives for initial training, then periodically re-mine hard negatives using the current model to refine the embedding space. The original DPR paper used BM25 negatives and in-batch negatives. More recent work like ANCE (Approximate Nearest Neighbor Negative Contrastive Estimation) demonstrates that self-mined hard negatives can substantially improve over BM25 negatives, especially later in training when the model has become good enough that BM25 negatives are no longer challenging.

A typical dense retriever training pipeline proceeds as follows:

1. Initialize query and document encoders from a pre-trained language model (BERT, RoBERTa, or a domain-specific model)
2. Collect training data: query-positive document pairs plus negative documents using one of the strategies above
3. For each training batch, encode all queries and documents using the two encoders
4. Compute the contrastive loss using the similarity scores between query embeddings and all document embeddings in the batch
5. Backpropagate gradients through both encoders and update their weights
6. Periodically evaluate on a held-out validation set to monitor retrieval quality
7. Optionally, re-mine hard negatives using the current model state after every few thousand steps

The training process is computationally intensive because we need to encode many negatives per query to provide a strong training signal, and BERT-based encoders are large models. Techniques like in-batch negatives help by reusing computation across the batch, but training a high-quality dense retriever still typically requires substantial GPU resources and hours to days of compute time. Smaller, more efficient encoder architectures like MiniLM or DistilBERT can reduce training time substantially while maintaining competitive retrieval quality.

An important practical consideration is gradient flow through both encoders. Both the query encoder and the document encoder receive gradients during training, so both are updated simultaneously. This means the document embeddings change at every training step, which invalidates any cached embeddings. In practice, this is handled by recomputing document embeddings periodically (not after every step) or by freezing one encoder and only training the other, depending on the application.

Let's trace through a concrete numerical example to build intuition for how dense retrieval works from query input to ranked output. This example makes the abstract concepts concrete and shows exactly what happens at each step.

Setup: We have a three-dimensional toy embedding space. Query $q$ = "How do plants make food?" has been encoded to the vector $\mathbf{q} = [0.6, 0.5, 0.6]$. We have three candidate documents with pre-computed embeddings:

• Document A: "Photosynthesis turns light energy into chemical energy in plants." Embedding: $\mathbf{d}_A = [0.7, 0.6, 0.4]$
• Document B: "Plants produce glucose from carbon dioxide and sunlight." Embedding: $\mathbf{d}_B = [0.5, 0.7, 0.5]$
• Document C: "My grandmother makes delicious food using fresh vegetables." Embedding: $\mathbf{d}_C = [0.1, 0.2, 0.9]$

Documents A and B are about plant biology and photosynthesis; Document C is about cooking. The embeddings reflect this: A and B have high values in dimensions 1 and 2 (which we can think of as capturing "biology" and "plant science"), while C has a high value in dimension 3 (capturing "food/cooking").

Step 1: Normalize all embeddings. Before computing similarities, we normalize each vector to unit length:

Similarly normalizing: $\hat{\mathbf{d}}_A \approx [0.706, 0.606, 0.404]$, $\hat{\mathbf{d}}_B \approx [0.488, 0.683, 0.488]$, $\hat{\mathbf{d}}_C \approx [0.104, 0.208, 0.932]$.

Step 2: Compute cosine similarities (dot products of normalized vectors).

Step 3: Rank by similarity. The ranking is: Document A (0.984) > Document B (0.941) > Document C (0.737).

Step 4: Interpret the result. Despite Document C containing the word "food" (which appears in the query "How do plants make food?"), it ranks last. Documents A and B, which describe photosynthesis without using the words "make" or "food," rank first and second. The embedding space has captured the semantic intent of the query: we are asking about plant biology, and the model correctly identifies plant biology documents as most relevant, while recognizing that "making food" in the cooking sense is semantically different.

Now consider what BM25 would do. The query contains terms: "how," "do," "plants," "make," "food." Document A contains "plants" once. Document B contains "Plants" once. Document C contains "food" once and is about cooking with vegetables from a garden (semantically related to "plants" in a different sense). BM25 might rank Document C highest due to the "food" term match, which appears to be a highly specific query term. Document A and B each only match "plants," giving them lower BM25 scores. This illustrates the vocabulary mismatch problem directly: the lexical method is misled by surface-level matches, while the semantic method captures true intent.

The key insight from this numerical example is that dense retrieval's power comes entirely from the quality of the embedding space. The toy embeddings above were constructed to make the example clear, but real embedding spaces are 768-dimensional and learned from billions of text examples. The geometric relationships in those learned spaces capture fine-grained semantic associations that no human could engineer by hand.

Let's implement dense retrieval using the sentence-transformers library, which provides pre-trained bi-encoder models optimized for semantic similarity. The library wraps BERT-based models with sensible defaults for pooling and normalization, which makes it straightforward to get a working retrieval system up and running.

Each document is now represented as a 384-dimensional unit vector. The pre-normalization confirms that all embeddings have L2 norm of exactly 1.0, meaning we can use dot product as a drop-in replacement for cosine similarity at query time. In a production system, these embeddings would be stored in a vector index for efficient approximate nearest neighbor search over millions of documents.

Let's implement a function to retrieve documents for a query:

The retriever correctly identifies documents about neural network learning mechanisms: the first result explains hierarchical learning directly, the second covers gradient descent (the mechanism by which neural networks learn), and the third covers backpropagation (how gradients are computed to drive learning). None of these documents contain the exact phrase "neural networks learn" yet they are all semantically relevant. The model understands that learning in neural networks is achieved through gradient descent and backpropagation, not just through documents that use the word "learn."

Let's compare dense retrieval to a simple lexical approach on a query that demonstrates vocabulary mismatch:

Dense retrieval correctly identifies gradient descent (an optimization method), backpropagation (how the optimization is computed), and neural network learning (the context of deep learning), all without those exact terms appearing in the query. The lexical system returns documents with BM25 scores of 0.0 for most candidates because the query terms "deep," "learning," "optimization," and "methods" rarely appear verbatim in the corpus. When results do appear, they are based on coincidental term matches rather than semantic relevance.

Understanding the geometry of the embedding space helps build intuition for why dense retrieval works. Let's project the high-dimensional embeddings down to two dimensions using PCA to see how documents cluster:

The PCA projection reveals the clustering structure in the embedding space. Documents about neural network training mechanisms (gradient descent, backpropagation, hierarchical representations) cluster together in one region, while classical machine learning methods (SVMs, Random forests) cluster elsewhere, and specialized architectures (CNNs, RNNs, Transformers) form their own neighborhood. The query "How do neural networks learn?" falls naturally near the learning-mechanism cluster, which is exactly where dense retrieval would look for relevant documents.

In production systems, we typically process multiple queries simultaneously. Encoding queries one at a time misses the opportunity to parallelize computation on GPU hardware. Batch encoding is one of the key practical optimizations for real retrieval systems:

The matrix multiplication computes all query-document similarities simultaneously in a single operation. For a batch of $Q$ queries and $D$ documents with embedding dimension $d$, this requires $O(Q \cdot D \cdot d)$ operations performed in parallel on GPU, which is orders of magnitude faster than computing $Q \cdot D$ dot products sequentially. In practice, batching queries together is one of the simplest and most effective optimizations for production retrieval throughput.

Let's wrap everything into a clean, reusable search interface that demonstrates the full retrieval pipeline:

The separation between index() (offline) and search() (online) mirrors the architecture of production retrieval systems. The index step can be precomputed and cached to disk; in real systems it is typically stored in a vector database like FAISS, Pinecone, or Weaviate. The search step is the only latency-necessary path, and it reduces to a single transformer forward pass plus a matrix-vector multiply.

Understanding the key parameters of a dense retrieval system helps you tune it for your specific application:

• model_name_or_path: The pre-trained model weights for SentenceTransformer. Different models offer different trade-offs between speed (smaller models like MiniLM), accuracy (larger models like MPNet), and domain coverage (domain-specific fine-tuned models). Always evaluate on your target domain before choosing a model.
• normalize_embeddings: When set to True in model.encode(), produces unit-length vectors and enables dot product to equal cosine similarity. Most use cases should set this to True for consistent, interpretable similarity scores.
• top_k: The number of results to retrieve. In a RAG pipeline, this controls how much context the language model receives. Retrieving more candidates provides richer context but increases prompt length and can introduce irrelevant content. Values between 3 and 10 are typical, with larger values used when a reranker will further filter results.
• batch_size: The number of documents or queries to encode simultaneously. Larger batches use GPU memory more efficiently but require more memory. Typical values range from 32 to 256 depending on document length and GPU memory.
• Temperature ($\tau$): During training, controls the sharpness of the contrastive loss. Smaller values produce harder gradients; larger values produce softer distributions. At inference time, the metric is computed without temperature.

Dense retrieval has transformed information retrieval and forms the foundation of modern knowledge-intensive NLP systems. However, understanding its limitations is needed for effective deployment. No retrieval technology is universal, and dense retrieval has specific failure modes that practitioners must account for.

Dense retrievers require substantial training data to perform well on any given domain. Unlike BM25, which works out-of-the-box on any text collection using only the document corpus itself, dense models need labeled query-document pairs that match the target domain. A retriever trained on Wikipedia passages may perform poorly on legal contracts, scientific papers, clinical notes, or source code because those domains use different vocabulary and writing styles to express their semantic relationships. This domain sensitivity creates a "cold start" problem: organizations adopting dense retrieval for specialized applications need to either find a pre-trained model that covers their domain, collect domain-specific training data, or accept degraded performance. Collecting labeled training data is expensive and time-consuming, and for highly specialized domains, it may require domain experts who are scarce and expensive.

The computational and memory requirements of dense retrieval are significant and should not be underestimated at scale. Each document must be processed by a transformer encoder, and the resulting embeddings must be stored and indexed. For a corpus of 100 million documents with 384-dimensional embeddings in float32 precision, storage alone requires approximately 150 gigabytes, before any indexing structure overhead. While approximate nearest neighbor indices like HNSW or IVF-PQ can reduce search latency to milliseconds, they require additional memory and careful tuning. Sparse indices based on inverted lists, as used by BM25, typically require far less memory and can be searched efficiently with simple term lookups. The infrastructure investment for production dense retrieval, including GPU servers for encoding, vector databases for indexing, and replication for high availability, is substantially larger than for sparse retrieval.

Dense retrievers can also fail silently in ways that sparse retrievers do not. When BM25 returns poor results, the explanation is often immediately clear: the query terms simply do not appear in relevant documents. When a dense retriever fails, diagnosing the problem is much harder. The model might not have learned good representations for certain concepts, might confuse similar-sounding but semantically distinct entities (e.g., "Python the language" vs "Python the snake"), might not handle negation correctly (retrieving documents that say "A does not cause B" when the query asks "what causes A"), or might have distribution shift between its training data and the deployment domain. These opacity challenges make debugging and systematically improving dense retrieval systems more difficult than improving lexical systems.

Negation and logical constraints present a well-known challenge. Dense retrieval operates on semantic similarity, so "hotels without swimming pools" may retrieve documents about "hotels with swimming pools" because both texts are semantically about hotels and pools. The model lacks a mechanism to understand that the user explicitly wants to exclude certain properties. Similarly, numerical constraints ("papers from 2020 to 2022") and logical combinations ("A but not B") are poorly handled by similarity-based retrieval and often require post-retrieval filtering or more specialized architectures.

Despite these limitations, dense retrieval has had enormous impact on NLP and AI systems. Semantic matching at scale made several previously impractical capabilities feasible:

Open-domain question answering systems improved dramatically when they could retrieve passages by meaning rather than keywords. Systems like RAG, FiD (Fusion-in-Decoder), and REALM all rely on dense retrieval as their backbone, achieving human-competitive performance on open-domain QA benchmarks by grounding generation in retrieved evidence. This was a major milestone in moving NLP from narrow task-specific systems to general knowledge-intensive question answering.

Retrieval-Augmented Generation systems, which we introduced in the previous chapters, depend critically on dense retrieval. The ability to find semantically relevant passages allows LLMs to answer questions about events after their knowledge cutoff, provide responses grounded in specific private documents, and reason over large knowledge bases without memorizing everything in their parameters. Every major commercial LLM assistant now uses some form of retrieval augmentation, and dense retrieval is the workhorse of the retrieval component in most of these systems.

Semantic search products from Google, Microsoft Bing, and other major search providers now incorporate dense retrieval techniques to handle conversational queries, long-form questions, and semantic information needs that were previously poorly served by keyword search. The transition from keyword search to semantic search at commercial scale represents one of the most significant applied NLP developments of the 2020s.

Cross-lingual applications became far more practical because multilingual dense models can encode meaning across languages, letting search systems where queries and documents may be in different languages. A user querying in English can retrieve relevant passages from documents in Spanish, French, or Japanese, as long as the multilingual encoder has learned to map semantically equivalent content in different languages to similar regions of the embedding space.

The combination of transformer pre-training, contrastive learning for retrieval, and approximate nearest neighbor search established a new paradigm for information access that goes far beyond traditional search. Dense retrieval combines representation learning with information retrieval, an area that continues to drive active research and development. The next chapter examines contrastive learning for retrieval: the training objectives, loss functions, and hard negative mining techniques that determine the quality of the learned embedding space.

Dense retrieval represents a major change from lexical to semantic matching in information retrieval. Rather than counting term overlaps, dense retrievers encode queries and documents into continuous vector spaces where similarity reflects semantic relatedness. The core value proposition is overcoming vocabulary mismatch: finding relevant documents regardless of whether they use the same specific words as the query.

The bi-encoder architecture enables scalable dense retrieval by separating query and document encoding. Documents are pre-computed and indexed offline, requiring only a single query encoding at search time. This design reduces query-time computation from millions of transformer forward passes to just one, letting practical retrieval over corpora with millions or billions of documents. The trade-off is that independent encoding sacrifices some modeling power compared to cross-encoders that jointly process query-document pairs, which is why dense bi-encoders are used for first-stage retrieval while cross-encoders serve as second-stage rerankers.

Embedding similarity metrics, particularly cosine similarity and dot product on normalized embeddings, quantify semantic relatedness between query and document vectors. The equivalence of these metrics under L2 normalization allows production systems to pre-normalize embeddings once at index time and use fast dot products at query time. The choice of metric should match the model's training objective.

Dense and sparse retrieval have complementary strengths that are best exploited by hybrid systems. Dense retrieval handles semantic matches and synonyms well, including conversational queries. Sparse retrieval provides interpretable exact-term matching with strong rare-term handling and zero-shot domain transfer. Hybrid retrieval combining both approaches consistently outperforms either approach alone on diverse retrieval benchmarks.

Training dense retrievers requires query-document pairs and careful negative sampling. The contrastive learning objective pushes the model to maximize similarity between relevant pairs while minimizing similarity with negatives. The quality of negatives significantly impacts the learned embedding space: hard negatives mined from BM25 or from the model itself produce substantially better representations than random negatives.

The next chapter examines contrastive learning for retrieval, including the training objectives, loss functions, and techniques that produce effective dense retrieval models from raw question-answer data.

Ready to test your understanding? Take this quick quiz to reinforce what you've learned about dense retrieval and semantic matching.