Vector Space Model & TF-IDF (1968) › Legacy: The Geometric Turn in Language AI

The Vector Space Model established a geometric representation for document retrieval. Terms became dimensions, documents and queries became weighted points, and similarity became a numerical operation.

Later word embeddings and neural language models also use vectors, although their dimensions are learned features rather than vocabulary terms. Cosine similarity remains common for comparing embeddings, while transformers perform additional learned operations over sequences of vectors. The shared mathematics should not obscure the differences between sparse term counts and contextual representations.

TF-IDF assigns importance from local frequency and collection-wide rarity. Attention weights are learned for a different purpose and depend on context, so they are not a neural version of IDF. Both mechanisms weight information, but they do so with different inputs and objectives.

SMART also helped make retrieval an empirical discipline. Researchers could vary weighting functions, test them on shared collections, and compare ranking metrics. Later learning-to-rank and neural retrieval methods extended that experimental framework with learned parameters.

Sparse vector methods remained common in information retrieval for decades and still serve as baselines or production components. Their behavior is interpretable, their storage can be efficient, and they require no supervised training data.

The model showed that term statistics and geometric comparison could support useful document rankings without a hand-built symbolic account of each document's meaning. It also made alternative weighting schemes testable on shared data.