Embedding Models: Architecture, Pooling & Selection › Embedding Model Selection

Choosing the right embedding model for your RAG pipeline is one of the highest-impact decisions you will make. A good model makes retrieval accurate; a poor choice creates a performance ceiling that no amount of downstream engineering can overcome. The decision involves multiple competing considerations: retrieval quality, inference cost, context length, language coverage, and operational simplicity.

The embedding model selection problem is analogous to choosing a foundation for a building. The foundation determines what structures are possible above it. A weak foundation limits everything built on top, no matter how carefully the upper floors are constructed. An overly expensive foundation might provide unnecessary strength at a cost that makes the whole project impractical. You want the foundation that is exactly right for the structure you intend to build, evaluated in the context of your actual requirements.

The Massive Text Embedding Benchmark (MTEB) by Muennighoff et al. (2023) is the standard benchmark for evaluating embedding models. It covers a wide range of tasks across multiple domains and languages. This provides a complete picture of model capabilities. The tasks it evaluates include the following:

• Retrieval: Finding relevant documents given a query, which is the most directly relevant task for RAG applications.
• Semantic Textual Similarity (STS): Scoring how similar two sentences are, which tests whether the model's similarity scores are well-calibrated.
• Classification: Using embeddings as features for text classification, which tests whether the embedding captures task-relevant semantic content.
• Clustering: Grouping similar documents together, which tests the global structure of the embedding space.
• Pair Classification: Determining if two texts are paraphrases, entailments, or contradictions.
• Reranking: Reordering candidate documents by relevance to a query.
• Summarization: Evaluating summary quality via embedding similarity.

The MTEB leaderboard (available at huggingface.co/spaces/mteb/leaderboard) ranks models across these tasks and provides aggregated scores. For RAG applications, the retrieval score matters most because it most directly reflects how well the model will perform at finding relevant passages. Good performance on STS and clustering suggests the model has learned a well-organized semantic space overall, which is a positive indicator even for retrieval tasks not directly evaluated in MTEB.

However, you should treat MTEB scores as a starting point, not a definitive answer. MTEB is computed on general-purpose benchmarks in a handful of domains. If your corpus contains specialized content such as legal contracts, medical literature, scientific papers, or software documentation, the benchmark scores may not accurately predict performance on your data. The model that ranks first on MTEB might rank second or third on your specific domain if the benchmark does not cover similar content.

When choosing an embedding model for a RAG system, consider these factors in roughly this order of importance, adjusting based on your specific constraints and requirements.

Retrieval quality on your domain is the primary criterion. MTEB scores are computed on general-purpose benchmarks. If you're working in a specialized domain such as legal, medical, or scientific content, benchmark scores may not reflect actual performance. Always evaluate candidate models on a sample of your own data before committing to a model in production. Even a small evaluation set of 50 to 100 query-document pairs representative of your use case will give you far more reliable signal than public benchmarks alone.

Language support is non-negotiable if your corpus includes non-English content. You need a multilingual model such as multilingual-e5-large, bge-m3, or paraphrase-multilingual-mpnet-base-v2. English-only models will fail silently on non-English text: they will produce embeddings, but these embeddings will not be comparable across languages in a useful way, and retrieval quality will degrade substantially. If your corpus mixes languages, even a small fraction of non-English content can create confusing retrieval behavior if the model was not trained multilingually.

Context length determines the maximum size of a document chunk the model can process. Standard BERT-based models handle 512 tokens. Many modern models support 8,192 tokens (like nomic-embed-text-v1.5 and gte-large-en-v1.5) or even longer. Your chunk sizes from the document chunking stage should fit comfortably within the model's context window. If your chunks exceed the model's limit, the text will be silently truncated, losing information without any warning or error.

Embedding dimension and inference cost matter enormously at scale. For a corpus of 10 million documents, the difference between a 22M-parameter model and a 7B-parameter model is enormous in compute cost. Consider whether the quality improvement justifies the cost for your application, and whether you have the infrastructure to support larger models at your required throughput.

Matryoshka support gives you flexibility to trade quality for speed at serving time. If storage costs are a concern, or if you want to use a two-stage retrieval approach, models with Matryoshka embeddings give you this option without retraining.

Instruction support allows task-aware models to improve retrieval quality by conditioning the embedding on the retrieval intent. This is especially useful when the same corpus is used for different types of queries, or when the gap between query style and document style is large.

Based on the trade-offs above, embedding models fall into three practical tiers for RAG applications.

Lightweight models (under 50M parameters, 384 dimensions) include models like all-MiniLM-L6-v2. They offer fast inference and small vector sizes, making them suitable for prototyping, low-resource deployments, or applications where latency matters more than peak accuracy. Encoding millions of documents is fast and cheap, and you can iterate quickly on other parts of your pipeline. The trade-off is lower retrieval quality, particularly on specialized domains or subtle semantic distinctions.

Mid-range models (100M to 350M parameters, 768 to 1024 dimensions) include models like bge-base-en-v1.5, gte-large-en-v1.5, and e5-large-v2. These represent the best quality-cost trade-off for most production systems. They offer strong retrieval quality on general-purpose and many specialized domains, while remaining affordable to run on GPUs or even CPUs for moderate-scale corpora.

Heavy models (1B or more parameters, 1536 to 4096 dimensions) include models like gte-Qwen2-1.5B-instruct and E5-Mistral-7B-instruct. These push the quality frontier but require GPU inference and produce large vectors. They are appropriate when retrieval quality takes priority over compute cost, or when serving a smaller, high-value corpus.

Let's walk through a practical evaluation workflow that you can adapt for your own use case. We'll compare two models on a small retrieval task using Mean Reciprocal Rank (MRR) and Recall at $k$ as our metrics. These two metrics capture complementary aspects of retrieval quality and together give a complete picture of model performance.

The high scores in this example confirm that both models successfully retrieve the relevant documents for these clear, well-formed queries. In practice, you want at least 50 to 100 query-document pairs representative of your actual use case, including difficult queries where similar but incorrect documents might be retrieved, queries that use different vocabulary than the documents, and queries that require understanding of domain-specific terminology.

MRR (Mean Reciprocal Rank) measures how high the first relevant document appears in the ranked list. An MRR of 1.0 means the correct document is always ranked first; an MRR of 0.5 means it appears on average at position 2. MRR is most informative when you retrieve only a small number of candidates (for example, if you pass only the top-3 results to the LLM). Recall@k measures what fraction of all relevant documents appear in the top-$k$ results. If a query has two relevant documents and both appear in the top 3, Recall@3 is 1.0 for that query. Recall@k is most informative when you want to ensure complete coverage: if you miss any relevant document, the LLM cannot use that information to answer the question.

The key parameters for SentenceTransformer models are:

• model_name_or_path: The Hugging Face model ID (e.g., "BAAI/bge-base-en-v1.5") or local path to the model directory.
• normalize_embeddings: If True during encoding, normalizes output vectors to unit length, letting cosine similarity computation via a simple dot product.

Not all embedding models need to run locally. Several providers offer embedding models as API services, which can be convenient for prototyping or for small-scale applications where managing GPU infrastructure is not practical.

OpenAI offers text-embedding-3-small (1536 dimensions) and text-embedding-3-large (3072 dimensions), both supporting Matryoshka-style dimension reduction via the dimensions parameter. Cohere offers embed-english-v3.0 and multilingual variants, with distinct input type parameters (search_query vs. search_document) that function similarly to instruction-conditioned models. Google provides Vertex AI text embeddings optimized for Google Cloud deployments. Voyage AI offers domain-specific models specialized for code, legal text, and financial documents.

API models trade control and cost predictability for operational convenience. For production RAG systems with large corpora, the per-token cost of API embeddings adds up quickly: at typical API pricing, embedding a 10-million-document corpus can cost hundreds of dollars or more, and re-embedding when the model changes costs the same again. Open-source models you run locally avoid these risks, though they introduce operational complexity in exchange.

A practical concern with API embedding models is vendor lock-in. Your entire vector index is tied to a specific model version from a specific provider. If the provider deprecates the model, changes pricing substantially, or experiences an outage, your entire retrieval system is affected. This has happened before with older OpenAI embedding model versions, requiring teams to re-embed their entire corpora. Open-source models you run locally avoid these risks entirely.