Dense Retrieval: Semantic Search & Bi-Encoder Implementation › Key Parameters

Understanding the key parameters of a dense retrieval system helps you tune it for your specific application:

• model_name_or_path: The pre-trained model weights for SentenceTransformer. Different models offer different trade-offs between speed (smaller models like MiniLM), accuracy (larger models like MPNet), and domain coverage (domain-specific fine-tuned models). Always evaluate on your target domain before choosing a model.
• normalize_embeddings: When set to True in model.encode(), produces unit-length vectors and enables dot product to equal cosine similarity. Most use cases should set this to True for consistent, interpretable similarity scores.
• top_k: The number of results to retrieve. In a RAG pipeline, this controls how much context the language model receives. Retrieving more candidates provides richer context but increases prompt length and can introduce irrelevant content. Values between 3 and 10 are typical, with larger values used when a reranker will further filter results.
• batch_size: The number of documents or queries to encode simultaneously. Larger batches use GPU memory more efficiently but require more memory. Typical values range from 32 to 256 depending on document length and GPU memory.
• Temperature ($\tau$): During training, controls the sharpness of the contrastive loss. Smaller values produce harder gradients; larger values produce softer distributions. At inference time, the metric is computed without temperature.