Product Quantization: Vector Compression for ANN Search › Key Parameters

Understanding PQ's parameters helps you make good configuration decisions. The parameters do not have universal optimal values; the right choices depend on your data, your memory budget, and your latency and recall requirements. A good approach is to start with the commonly recommended defaults, measure recall on a representative query set, and then adjust from there.

The key parameters for Product Quantization with FAISS are:

• M: Number of subspaces (subvector segments). At fixed nbits, more subspaces produce a longer code, less compression, and generally higher recall. For 768-dimensional embeddings, values between 32 and 96 are common starting points, but the choice should be reported together with nbits or the resulting bytes per vector.
• nbits: Number of bits per subcode, which determines the codebook size as $K = 2^{\text{nbits}}$. Using nbits=8 gives 256 centroids per subspace and one byte of storage per subspace. Using nbits=4 halves the storage but reduces the codebook from 256 to 16 centroids, increasing quantization error.
• nlist: Number of IVF coarse clusters (Voronoi cells) in IVFPQ. More clusters means finer partitioning and faster search, but requires more training data and longer training time. A common rule of thumb is $\text{nlist} \approx \sqrt{N}$ where $N$ is the number of database vectors.
• nprobe: Number of IVF cells to search at query time. Higher values improve recall at the cost of search speed. The recall vs. nprobe curve (shown in the plots above) lets you choose the operating point that fits your latency budget.
• Training set size: The number of training vectors used to learn the codebooks. At least a few hundred times $K$ per subspace is recommended for reliable k-means convergence. For nbits=8 ($K = 256$), this means at least 50,000 training vectors in practice.