Embedding Models: Architecture, Pooling & Selection › Embedding Model Architecture and Selection

In the previous chapters on dense retrieval and contrastive learning, we established why we need dense vector representations and how models learn to place semantically similar texts near each other in vector space. We explored the mechanics of contrastive loss functions and saw how training pairs can sculpt a continuous geometry of meaning. However, we did not address a necessary question: how does a transformer, which produces a sequence of token-level hidden states, yield a single vector that represents an entire sentence or paragraph? The answer lies in the architecture and design choices of embedding models, the specialized systems that convert variable-length text into fixed-dimensional vectors suitable for similarity search.

Embedding models sit at the heart of every RAG pipeline. The quality of your retrieval, and therefore the quality of your generated answers, depends directly on how well your embedding model captures semantic meaning. A poorly chosen or misconfigured embedding model will return irrelevant chunks no matter how sophisticated the rest of your pipeline is. You could have the most advanced reranker, the most carefully designed prompts, and the most capable generative LLM in the world, but if your retriever returns the wrong passages, none of that sophistication can compensate. The embedding model is the foundation on which everything else stands.

Think of an embedding model as a very sophisticated filing system. When you file a document, you decide what folder it belongs in, and later you can retrieve it by looking in the right folder. An embedding model does something far more fine-grained: it places each piece of text at a precise coordinate in a high-dimensional space, where the coordinates encode the text's meaning. Documents about similar topics land near each other; documents about unrelated topics land far apart. When a query arrives, you convert it to coordinates in the same space and look for nearby documents. The quality of this "filing" determines everything about your retrieval system's accuracy.

This chapter examines embedding models from the inside out. We'll start with the architectural foundations and the non-trivial problem of distilling a sequence into a single vector. Then we'll explore the pooling strategies that perform this distillation, examining each one's mathematical properties and practical implications. We'll discuss the role of embedding dimensionality, including the elegant Matryoshka technique that allows a single model to serve multiple dimensionality requirements. We'll work through a concrete numerical example that makes these concepts tangible. Finally, we'll cover how to select and evaluate a model before deploying it for your use case, including limitations you need to be aware of.