Michael Brenndoerfer | Data & AI, Private Equity, Technology

Michael Brenndoerfer

Vector Similarity Search: Metrics & Approximate Methods › Worked Example: Metrics in Action

Michael BrenndoerferJanuary 25, 20268 min read

Language AI HandbookAll chapters

Part 44 of 67Retrieval-Augmented Generation

Chapter 338Vector Similarity Search: Metrics & Approximate Methods › Worked Example: Metrics in Action

Reading section 5 of 11 · Worked Example: Metrics in Action · 8 minSection 5 of 11 · 8 min

Section 5 of 11 · 8 min read

Vector Similarity Search: Metrics & Approximate Methods › Worked Example: Metrics in ActionOne section · Sections in this chapter5 / 11 ↓

Sections in this chapter11

Reading tools Definitions & inline links

Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.

Article links

Make inline references clickable

Let's make these metrics concrete with a small example that illustrates the key differences between direction-based and position-based measures. Suppose we have a query vector and four document vectors, all in $\mathbb{R}^3$ for visualization simplicity. The low dimensionality lets us reason about the geometry directly, and the principles we observe generalize straightforwardly to the hundreds of dimensions used in real embeddings.

\mathbf{q} = [1.0, 2.0, 0.5], \quad \mathbf{v}_1 = [1.1, 1.9, 0.6], \quad \mathbf{v}_2 = [3.0, 6.0, 1.5], \quad \mathbf{v}_3 = [0.0, 1.0, 3.0], \quad \mathbf{v}_4 = [-1.0, -2.0, -0.5]

These four vectors were chosen deliberately to illustrate specific geometric relationships. Notice that $\mathbf{v}_2 = 3 \times \mathbf{q}$ , so it points in exactly the same direction but has 3 times the magnitude. This makes it a perfect test case for the distinction between direction-based and position-based metrics: any direction-based metric will consider $\mathbf{v}_2$ identical to $\mathbf{q}$ , while any position-based metric will see them as far apart. Meanwhile, $\mathbf{v}_4 = -\mathbf{q}$ , pointing in the exact opposite direction, which should produce the worst possible cosine similarity. The vector $\mathbf{v}_1$ is the subtlest case: it is close to $\mathbf{q}$ in both direction and magnitude, differing only by small perturbations in each component. Finally, $\mathbf{v}_3$ points in a substantially different direction, with a large component in the third dimension where $\mathbf{q}$ has only a small value.

Before computing anything, we can reason geometrically about what to expect. Cosine similarity between $\mathbf{q}$ and $\mathbf{v}_2$ should be 1.0 (identical direction), while cosine similarity between $\mathbf{q}$ and $\mathbf{v}_4$ should be $-1.0$ (opposite direction). Meanwhile, $\mathbf{v}_1$ is close to $\mathbf{q}$ in both direction and magnitude, so it should also have high cosine similarity, though not a perfect 1.0.

Euclidean distance between $\mathbf{q}$ and $\mathbf{v}_1$ should be small (they are nearby), while the distance between $\mathbf{q}$ and $\mathbf{v}_2$ will be large despite their identical directions (because $\mathbf{v}_2$ is far away in absolute terms). This is the signature behavior that distinguishes position-based from direction-based metrics.

In[5]:

Code

import numpy as np

# Define query and document vectors
q = np.array([1.0, 2.0, 0.5])
v1 = np.array([1.1, 1.9, 0.6])
v2 = np.array([3.0, 6.0, 1.5])
v3 = np.array([0.0, 1.0, 3.0])
v4 = np.array([-1.0, -2.0, -0.5])

vectors = {"v1": v1, "v2": v2, "v3": v3, "v4": v4}


def cosine_similarity(a, b):
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))


def euclidean_distance(a, b):
    return np.linalg.norm(a - b)


def manhattan_distance(a, b):
    return np.sum(np.abs(a - b))


def dot_product(a, b):
    return np.dot(a, b)


# Calculate metrics for all vectors
results = []
for name, v in vectors.items():
    results.append(
        {
            "name": name,
            "cos": cosine_similarity(q, v),
            "dot": dot_product(q, v),
            "euc": euclidean_distance(q, v),
            "man": manhattan_distance(q, v),
        }
    )

Out[6]:

Console

Vector     Cosine Sim   Dot Product   Euclidean   Manhattan
------------------------------------------------------------
v1              0.997         5.200       0.173       0.300
v2              1.000        15.750       4.583       7.000
v3              0.483         3.500       2.872       4.500
v4             -1.000        -5.250       4.583       7.000

Out[7]:

Visualization

Arrow diagram showing vectors q, v1, v2, and v3 from the origin with dashed L2 distance annotations. — 2D projection illustrating the difference between angle-based (cosine) and position-based (Euclidean) metrics. Vector v2 points in the same direction as q (angle 0, cosine similarity 1.0) but is far away in absolute distance, while v1 is close in both direction and position. The dashed lines show the Euclidean distances from the query tip to each vector tip.

The results confirm our predictions. By cosine similarity, $\mathbf{v}_2$ scores a perfect 1.0 (same direction as the query) while $\mathbf{v}_4$ scores $-1.0$ (opposite direction). But by Euclidean distance, $\mathbf{v}_1$ is the closest neighbor (only 0.1732 away), while $\mathbf{v}_2$ is much farther despite its perfect directional alignment.

This example highlights the core difference between direction-based metrics (cosine) and position-based metrics (Euclidean). Cosine similarity asks, "Which document is most topically aligned with the query?" Euclidean distance asks, "Which document is closest to the query in the embedding space?" For normalized vectors these questions have the same answer, but for unnormalized vectors they can diverge significantly.

Now let's see what happens when we normalize the vectors:

In[8]:

Code

# Normalize all vectors to unit length
q_norm = q / np.linalg.norm(q)
vectors_norm = {name: v / np.linalg.norm(v) for name, v in vectors.items()}

# Calculate metrics for normalized vectors
results_norm = []
for name, v in vectors_norm.items():
    results_norm.append(
        {
            "name": name,
            "cos": cosine_similarity(q_norm, v),
            "dot": dot_product(q_norm, v),
            "euc": euclidean_distance(q_norm, v),
        }
    )

Out[9]:

Console

Vector     Cosine Sim   Dot Product   Euclidean
------------------------------------------------
v1              0.997         0.997       0.076
v2              1.000         1.000       0.000
v3              0.483         0.483       1.017
v4             -1.000        -1.000       2.000

After normalization, cosine similarity and dot product produce identical values, confirming the mathematical relationship we derived earlier: for unit vectors, $\mathbf{a} \cdot \mathbf{b} = \cos\theta$ . And the rankings from all three metrics now agree: the vectors with higher cosine similarity also have smaller Euclidean distance, exactly as the identity $d_{\text{L2}}^2 = 2 - 2\cos\theta$ predicts. Notice in particular that $\mathbf{v}_2$ , which was far from the query before normalization, now sits at exactly the same point as the normalized query, with Euclidean distance 0.0. Normalization collapsed the magnitude information and left only direction.

This is the practical reason why normalizing embeddings before indexing is so common: it makes the choice of metric irrelevant. Once all vectors lie on the unit sphere, cosine similarity, dot product, and Euclidean distance all induce the same ranking over candidates. You can then choose whichever metric your library computes most efficiently, typically the dot product, without worrying about whether it matches the conceptual notion of similarity your model was trained with.

Visualizing the Metric DistributionLink Copied

Another way to understand how metrics differ is to examine their distribution over a realistic embedding collection. When you compute all pairwise similarities in a real embedding collection, the distribution reveals the geometry of the embedding space: how tight the clusters are, how much separation exists between topics, and what similarity values to expect for relevant versus irrelevant documents.

Out[10]:

Visualization

Histogram of cosine similarity values with vertical lines at the 95th and 99th percentiles. — Distribution of cosine similarities between a random query and 10,000 synthetic embeddings at dimension 128. The bulk of random pairs cluster near 0.0, while a small tail of semantically similar documents reaches toward 1.0. The vertical lines mark the 95th and 99th percentiles, showing the similarity thresholds at which only the top 5% and 1% of the collection would be retrieved.

The distribution shows how vector search works at scale. Most documents in the collection are only weakly similar to any given query: the bulk of cosine similarities cluster near zero. The relevant documents (simulated here as vectors near the query direction) form a distinct right tail. A retrieval system needs to identify this tail efficiently, which is why even small differences in the similarity metric can affect which documents make it into the retrieved set.