Contrastive Learning for Retrieval: InfoNCE & DPR Guide › The Contrastive Learning Framework

Contrastive learning is a form of metric learning. Instead of training a model to output a class label or predict the next token in a sequence, you train it to produce representations where a distance or similarity metric reflects the semantic relationship between inputs. The core philosophy is that you do not tell the model what something means in isolation; rather, you tell the model how things relate to each other, and you do so by showing both positive and negative examples of those relationships. For retrieval, this translates into a concrete objective: given a query $q$ and a relevant passage $p^+$, their embeddings should have high similarity, while a query and an irrelevant passage $p^-$ should have low similarity.

This framing sidesteps the need for explicit labels that describe what a passage is "about." You never need to assign a topic tag or a relevance score on a fixed scale. Instead, you only need to specify pairwise relationships: this query goes with this passage, and not with that one. The model's internal representations emerge organically from the pressure of having to satisfy many such relational constraints simultaneously.

The training data for contrastive retrieval consists of pairs $(q_i, p_i^+)$ where each query is associated with at least one known relevant passage. These positive pairs might come from various sources:

• Question-answer datasets: The passage containing the answer is the positive for that question.
• Click logs: A document that you clicked after issuing a search query.
• Hyperlinks: The anchor text is a "query" and the linked page as the positive passage.
• Manual annotations: Human judges label query-document relevance.

Each of these sources carries different biases and varying levels of noise. Click logs are plentiful but noisy, since you sometimes click on irrelevant documents. Manual annotations are high-quality but expensive to collect at scale. Question-answer datasets offer a middle ground. This provides naturally occurring query-passage pairs where relevance is clearly defined by whether the passage contains the answer. Regardless of the source, the fundamental structure is the same: pairs of inputs that should end up close together in the learned embedding space.

The key challenge is that we also need negative examples, passages that are irrelevant to a given query. Where these negatives come from and how hard they are to distinguish from true positives strongly affects model quality. This is a recurring theme throughout the chapter. It reflects a core principle of contrastive learning: the model can only learn distinctions it is forced to make, and the quality of the negatives determines which distinctions the training process demands.

With the training data structure established, the next question is how to translate the intuition of "push positives together, pull negatives apart" into a differentiable objective that gradient descent can optimize. The path from intuition to formula begins with the similarity function.

Suppose you have a query $q$, one positive passage $p^+$, and $N$ negative passages $p_1^-, p_2^-, \ldots, p_N^-$. You encode all of them using your query encoder $E_Q$ and passage encoder $E_P$, then compute similarity scores:

$$s(q, p) = E_Q(q)^\top E_P(p)$$

where:

• $s(q, p)$: the scalar similarity score between query $q$ and passage $p$
• $E_Q(q)$: the dense vector embedding of the query produced by the query encoder
• $E_P(p)$: the dense vector embedding of the passage produced by the passage encoder
• $\top$: the transpose operation, computing the dot product between the two vectors

This equation captures the essence of how the dual-encoder architecture measures relevance. Each encoder independently maps its input text into a fixed-dimensional vector space, and the dot product between the resulting vectors is a proxy for semantic relatedness. The dot product is large and positive when the two vectors point in roughly the same direction, small or near zero when they are orthogonal (unrelated), and negative when they point in opposite directions. An alternative is cosine similarity, which normalizes both vectors to unit length before taking the dot product, but the principle remains the same.

This dot product (or cosine similarity) gives you a scalar score for each query-passage pair. The training objective then becomes: you want $s(q, p^+)$ to be much larger than any $s(q, p_i^-)$. The gap between the positive score and the negative scores is the signal that the loss function will exploit, and different loss functions formalize this requirement in different ways.