Dense Retrieval: Semantic Search & Bi-Encoder Implementation › Summary

Dense retrieval represents a major change from lexical to semantic matching in information retrieval. Rather than counting term overlaps, dense retrievers encode queries and documents into continuous vector spaces where similarity reflects semantic relatedness. The core value proposition is overcoming vocabulary mismatch: finding relevant documents regardless of whether they use the same specific words as the query.

The bi-encoder architecture enables scalable dense retrieval by separating query and document encoding. Documents are pre-computed and indexed offline, requiring only a single query encoding at search time. This design reduces query-time computation from millions of transformer forward passes to just one, letting practical retrieval over corpora with millions or billions of documents. The trade-off is that independent encoding sacrifices some modeling power compared to cross-encoders that jointly process query-document pairs, which is why dense bi-encoders are used for first-stage retrieval while cross-encoders serve as second-stage rerankers.

Embedding similarity metrics, particularly cosine similarity and dot product on normalized embeddings, quantify semantic relatedness between query and document vectors. The equivalence of these metrics under L2 normalization allows production systems to pre-normalize embeddings once at index time and use fast dot products at query time. The choice of metric should match the model's training objective.

Dense and sparse retrieval have complementary strengths that are best exploited by hybrid systems. Dense retrieval handles semantic matches and synonyms well, including conversational queries. Sparse retrieval provides interpretable exact-term matching with strong rare-term handling and zero-shot domain transfer. Hybrid retrieval combining both approaches consistently outperforms either approach alone on diverse retrieval benchmarks.

Training dense retrievers requires query-document pairs and careful negative sampling. The contrastive learning objective pushes the model to maximize similarity between relevant pairs while minimizing similarity with negatives. The quality of negatives significantly impacts the learned embedding space: hard negatives mined from BM25 or from the model itself produce substantially better representations than random negatives.

The next chapter examines contrastive learning for retrieval, including the training objectives, loss functions, and techniques that produce effective dense retrieval models from raw question-answer data.