Dense Retrieval: Semantic Search & Bi-Encoder Implementation › Worked Example: Semantic Similarity in Action

Let's trace through a concrete numerical example to build intuition for how dense retrieval works from query input to ranked output. This example makes the abstract concepts concrete and shows exactly what happens at each step.

Setup: We have a three-dimensional toy embedding space. Query $q$ = "How do plants make food?" has been encoded to the vector $\mathbf{q} = [0.6, 0.5, 0.6]$. We have three candidate documents with pre-computed embeddings:

• Document A: "Photosynthesis turns light energy into chemical energy in plants." Embedding: $\mathbf{d}_A = [0.7, 0.6, 0.4]$
• Document B: "Plants produce glucose from carbon dioxide and sunlight." Embedding: $\mathbf{d}_B = [0.5, 0.7, 0.5]$
• Document C: "My grandmother makes delicious food using fresh vegetables." Embedding: $\mathbf{d}_C = [0.1, 0.2, 0.9]$

Documents A and B are about plant biology and photosynthesis; Document C is about cooking. The embeddings reflect this: A and B have high values in dimensions 1 and 2 (which we can think of as capturing "biology" and "plant science"), while C has a high value in dimension 3 (capturing "food/cooking").

Step 1: Normalize all embeddings. Before computing similarities, we normalize each vector to unit length:

$$\|\mathbf{q}\| = \sqrt{0.6^2 + 0.5^2 + 0.6^2} = \sqrt{0.36 + 0.25 + 0.36} = \sqrt{0.97} \approx 0.985$$ $$\hat{\mathbf{q}} = \frac{[0.6, 0.5, 0.6]}{0.985} \approx [0.609, 0.508, 0.609]$$

Similarly normalizing: $\hat{\mathbf{d}}_A \approx [0.706, 0.606, 0.404]$, $\hat{\mathbf{d}}_B \approx [0.488, 0.683, 0.488]$, $\hat{\mathbf{d}}_C \approx [0.104, 0.208, 0.932]$.

Step 2: Compute cosine similarities (dot products of normalized vectors).

$$\begin{aligned} \text{sim}(\hat{\mathbf{q}}, \hat{\mathbf{d}}_A) &= (0.609)(0.706) + (0.508)(0.606) + (0.609)(0.404) \\ &= 0.430 + 0.308 + 0.246 = 0.984 \end{aligned}$$ $$\begin{aligned} \text{sim}(\hat{\mathbf{q}}, \hat{\mathbf{d}}_B) &= (0.609)(0.488) + (0.508)(0.683) + (0.609)(0.488) \\ &= 0.297 + 0.347 + 0.297 = 0.941 \end{aligned}$$ $$\begin{aligned} \text{sim}(\hat{\mathbf{q}}, \hat{\mathbf{d}}_C) &= (0.609)(0.104) + (0.508)(0.208) + (0.609)(0.932) \\ &= 0.063 + 0.106 + 0.568 = 0.737 \end{aligned}$$

Step 3: Rank by similarity. The ranking is: Document A (0.984) > Document B (0.941) > Document C (0.737).

Step 4: Interpret the result. Despite Document C containing the word "food" (which appears in the query "How do plants make food?"), it ranks last. Documents A and B, which describe photosynthesis without using the words "make" or "food," rank first and second. The embedding space has captured the semantic intent of the query: we are asking about plant biology, and the model correctly identifies plant biology documents as most relevant, while recognizing that "making food" in the cooking sense is semantically different.

Now consider what BM25 would do. The query contains terms: "how," "do," "plants," "make," "food." Document A contains "plants" once. Document B contains "Plants" once. Document C contains "food" once and is about cooking with vegetables from a garden (semantically related to "plants" in a different sense). BM25 might rank Document C highest due to the "food" term match, which appears to be a highly specific query term. Document A and B each only match "plants," giving them lower BM25 scores. This illustrates the vocabulary mismatch problem directly: the lexical method is misled by surface-level matches, while the semantic method captures true intent.

The key insight from this numerical example is that dense retrieval's power comes entirely from the quality of the embedding space. The toy embeddings above were constructed to make the example clear, but real embedding spaces are 768-dimensional and learned from billions of text examples. The geometric relationships in those learned spaces capture fine-grained semantic associations that no human could engineer by hand.