Embedding Models: Architecture, Pooling & Selection › Embedding Model Architectures

Modern embedding models share a common blueprint, but they differ in their choice of base encoder, training procedure, and output processing. Understanding these architectural families helps clarify what happens under the hood when you call a model's encode method and receive a vector back. The architectural family determines which pooling strategy is appropriate, how the model should be prompted, and what kinds of texts it can handle effectively.

Before we examine specific families, it is worth understanding the general structure that all embedding models share. Every embedding model takes a text string as input and produces a fixed-dimensional vector as output. Between these two endpoints, there are three main components. The tokenizer converts the raw text into a sequence of integer token IDs, handling subword splitting, special token insertion, and sequence padding or truncation. The transformer encoder then processes these token IDs through multiple layers of self-attention and feed-forward networks, creating a matrix of hidden states, one row per token. Finally, the pooling and projection layer collapses this matrix into a single vector and optionally applies normalization or dimensionality reduction. Each of these components has design choices with significant consequences for the quality and behavior of the resulting embeddings.

Sentence-BERT, introduced by Reimers and Gurevych (2019), established the dominant paradigm for embedding models. The core idea is elegant: take a pre-trained transformer encoder, add a pooling layer on top, and fine-tune the entire stack using siamese or triplet networks with a contrastive objective. What makes this approach so powerful is the combination of a strong pre-trained foundation, which provides rich contextual understanding of language built from billions of tokens of web text, with a training objective that explicitly organizes the output space for semantic comparison. The pre-training instills deep linguistic knowledge; the fine-tuning teaches the model how to express that knowledge as a geometry of meaning.

The architecture uses what's called a bi-encoder (or dual-encoder) pattern, which we encountered in the dense retrieval chapter. The query and the document are encoded independently by the same model, without any cross-attention between them. This independence is what makes bi-encoders fast at retrieval time: document embeddings can be precomputed and indexed once, and at query time only the query needs to be encoded and compared against the pre-indexed vectors. The key insight is that because the same encoder processes both queries and documents, both types of text are mapped into the same shared vector space. Proximity in this space reflects semantic similarity, regardless of whether the two texts being compared are both queries, both documents, or one of each. The shared space is what enables the model to match query intent to document content even when the specific words differ.

The forward pass for a single input proceeds through four key steps. First, tokenization splits the input text into subword tokens using the encoder's vocabulary, inserting special tokens like [CLS] at the beginning and [SEP] at the end. Second, encoding passes the token IDs through the transformer layers, creating contextualized token embeddings where each position's representation reflects the surrounding context. Third, pooling reduces the token embeddings to a single vector using a strategy like mean pooling. Fourth and optionally, the vector is L2-normalized to unit length, which simplifies similarity computation. Each of these steps involves design choices that affect the quality of the resulting embedding, and the pooling step in particular, which we will examine in depth shortly, is where much of the embedding model's character is determined.

Training SBERT-style models uses a siamese architecture during the training phase. Two copies of the same encoder (sharing all weights) process a pair of texts in parallel, creating two embeddings. A contrastive loss then trains these embeddings to be similar for semantically related pairs and dissimilar for unrelated pairs. The weight-sharing is needed: it ensures that the same text always produces the same embedding regardless of which "branch" of the siamese network processes it, which is a prerequisite for consistent similarity scoring. After training, only one copy of the encoder is needed for inference.

A limitation of standard embedding models is that they produce the same vector for a piece of text regardless of the intended task. The text "Python is a programming language" gets the same embedding whether you're doing topic classification, semantic search over technical documentation, or clustering discussion forum posts. This is a meaningful limitation because the aspects of a text that matter most depend entirely on how the embedding will be used. For a classification task, you might want the embedding to emphasize the category or topic. For a semantic search task, you might want it to emphasize the specific technical facts or claims that a query could ask about. For a clustering task, you might want it to emphasize the linguistic style and register of the text. A single, task-agnostic embedding cannot optimally serve all of these purposes simultaneously.

Task-aware models like Instructor (Su et al., 2023) address this limitation by prepending a natural language task instruction to each input. Instead of encoding just the text, you encode something like: "Represent the science document for retrieval: Python is a programming language." The instruction conditions the model to produce embeddings optimized for the specified task. By seeing the instruction as part of the input sequence, the model can attend to the instruction when building the contextual representations of the main text, effectively shifting what aspects of meaning it emphasizes.

This approach does not require architectural changes compared to standard embedding models: it uses the transformer's ability to attend across the full input, including the instruction prefix. The model is trained on diverse tasks with diverse instructions, learning to shift its embedding space based on the instruction context. The result is a single model that behaves like many specialized models, adapting its output representations to the task at hand simply by reading a natural language instruction. For a RAG system serving multiple query types, for example factual lookup, comparison, and definition retrieval, you can use different instructions to bias the embeddings appropriately for each type.

While not strictly single-vector embedding models, late interaction architectures like ColBERT deserve mention because they represent a middle ground between bi-encoders and cross-encoders. The basic trade-off in retrieval model design is between expressiveness and efficiency. Bi-encoders are efficient because they compress each text to a single vector, but this compression sacrifices fine-grained token-level matching. Two passages that contain the same key phrase but in very different contexts might receive similar embeddings from a bi-encoder, even though their relevance to a specific query differs substantially. Cross-encoders are expressive because they perform full token-level cross-attention between a query and a document, but this means the document cannot be pre-encoded independently of the query, making them too slow for first-stage retrieval over large corpora.

Late interaction models handle this trade-off by retaining more information than bi-encoders while remaining far more efficient than cross-encoders. Instead of compressing each text into a single vector, ColBERT retains all token-level embeddings and computes similarity using a MaxSim operation: for each query token, find its maximum cosine similarity to any document token, then sum these maximums across all query tokens. This mechanism allows ColBERT to capture fine-grained lexical and semantic matches between individual query terms and document terms, something that a single-vector comparison cannot achieve.

The MaxSim operation is defined as:

$$\text{Score}(q, d) = \sum_{i \in q} \max_{j \in d} \cos(\mathbf{h}_i^q, \mathbf{h}_j^d)$$

where:

• $q$: the query text with token positions $i$
• $d$: the document text with token positions $j$
• $\mathbf{h}_i^q$: the embedding of query token $i$
• $\mathbf{h}_j^d$: the embedding of document token $j$
• $\cos(\cdot, \cdot)$: cosine similarity between two token embeddings

The key insight here is that every query token gets to "find" its best match in the document, so even rare or domain-specific terms that might be washed out in a mean-pooled embedding can contribute to the overall relevance score. This preserves more fine-grained information than single-vector approaches while remaining more efficient than full cross-attention, since document token embeddings can still be precomputed and indexed. We'll see how ColBERT fits into retrieval pipelines when we discuss reranking later in this part.

Recent embedding models have pushed performance significantly through several complementary innovations. These advances reflect a broader trend in the field: rather than relying on a single breakthrough technique, the best embedding models combine multiple strategies to achieve state-of-the-art results. Understanding these innovations helps you evaluate available embedding models and anticipate where the field is heading.

The first major innovation is the use of larger and better base models. Early embedding models used BERT-base (110M parameters) or BERT-large (340M parameters) as their backbone. More recent models have moved to much larger encoders or even decoder-based models. Models like E5-Mistral use a decoder-only LLM (Mistral 7B, with 7 billion parameters) as the backbone, applying special pooling to extract embeddings. The intuition here is that larger language models develop richer internal representations of language during pre-training on vastly larger datasets. These richer representations provide a better starting point for embedding fine-tuning, and they can encode more fine-grained semantic distinctions than smaller models.

The second major innovation is multi-stage training. The GTE, BGE, and E5 model families use a progressive training pipeline with distinct stages. The first stage uses large-scale weakly-supervised data: for example, title-body pairs from web pages, question-answer pairs from forums, or paraphrase pairs from multilingual corpora. This teaches the model broad notions of textual relevance from massive, automatically collected data. The second stage fine-tunes on curated, human-annotated data where quality is much higher but quantity is lower. This refines the model's understanding using high-quality human judgments about relevance. An optional third stage performs knowledge distillation from a larger teacher model into a smaller student, compressing performance from a model too large for practical inference into one that can be deployed efficiently.

Third is Matryoshka representation learning, which trains embeddings so that any prefix of the vector is itself a useful embedding. This technique, which we will examine in detail later in this chapter, allows a single model to serve multiple dimensionality requirements by simply truncating its output vector at different lengths. Fourth is unified models for multiple tasks: models like GTE-Qwen2 handle embedding, retrieval, reranking, and classification within a single architecture, using natural language instructions to switch between modes. This unification simplifies deployment because a single model can serve multiple roles in the pipeline, reducing the number of models you need to manage.