Term Frequency › L2-Normalized Term Frequency Vectors

The most mathematically principled normalization strategy places every document vector on the unit sphere, making length-invariant comparison a simple dot product.

To normalize a vector to unit length, divide it by its Euclidean norm:

$$\hat{\mathbf{v}}_d = \frac{\mathbf{v}_d}{\|\mathbf{v}_d\|_2}$$

where:

• $\mathbf{v}_d \in \mathbb{R}^{|V|}$: the raw (or log-scaled) term frequency vector for document $d$, with one entry per vocabulary term
• $\|\mathbf{v}_d\|_2$: the Euclidean (L2) norm of the vector, computed as

$$\|\mathbf{v}_d\|_2 = \sqrt{\sum_{t \in V} \text{count}(t, d)^2}$$

• $\hat{\mathbf{v}}_d$: the L2-normalized document vector, which satisfies $\|\hat{\mathbf{v}}_d\|_2 = 1$

After L2 normalization, every document vector lies on the unit hypersphere in $\mathbb{R}^{|V|}$.

Why L2 normalization? Two key properties make it highly useful.

First, length invariance: if you double all counts in a document (as would happen if you copy-pasted it twice), the unnormalized vector doubles, but the normalized vector remains unchanged. The representation captures relative word proportions, not absolute volumes. This property is essential for fair comparison across documents of different lengths.

Second, cosine similarity simplification: the cosine similarity between two documents measures the angle between their vectors:

$$\cos(\mathbf{v}_d, \mathbf{v}_{d'}) = \frac{\mathbf{v}_d \cdot \mathbf{v}_{d'}}{\|\mathbf{v}_d\|_2 \cdot \|\mathbf{v}_{d'}\|_2}$$

When both vectors are already L2-normalized (so $\|\hat{\mathbf{v}}_d\|_2 = \|\hat{\mathbf{v}}_{d'}\|_2 = 1$), this simplifies to:

$$\cos(\hat{\mathbf{v}}_d, \hat{\mathbf{v}}_{d'}) = \hat{\mathbf{v}}_d \cdot \hat{\mathbf{v}}_{d'} = \sum_{t \in V} \hat{v}_{d,t} \cdot \hat{v}_{d',t}$$

The division by norms disappears entirely. This is extremely convenient: once vectors are normalized, you can compute document similarity with a single dot product, which is highly optimized in numerical libraries. For a retrieval system comparing a query against millions of documents, this simplification translates directly into speed.

L2 normalization vs. L1 normalization. An alternative is L1 normalization, which divides by the sum of absolute values rather than the Euclidean norm:

$$\|\mathbf{v}_d\|_1 = \sum_{t \in V} |\text{count}(t, d)| = \sum_{t \in V} \text{count}(t, d)$$

L1 normalization produces a probability distribution over terms: each entry $\hat{v}_{d,t} = \text{count}(t, d) / |d|$ is the fraction of document $d$'s words that are term $t$. This is equivalent to dividing by document length and gives the relative frequency interpretation. L1 normalization is more interpretable (each weight is literally a proportion), while L2 normalization gives better geometric properties for similarity computation. Both are valid; the choice depends on whether you prioritize interpretability or computational convenience.