Product Quantization: Vector Compression for ANN Search › Limitations and Practical Implications

Product Quantization is not a free lunch. Understanding its limitations helps you anticipate where it will work well and where it will require additional engineering to meet accuracy requirements.

The core limitation is the accuracy-compression trade-off: any time you compress, you lose information, and that information loss translates to missed neighbors. For applications where recall below 95% is unacceptable, PQ alone may not suffice, and you will need to pair it with a post-processing reranking step using the original full-precision vectors. This two-stage architecture, retrieve with PQ and then rerank with exact distances, is now standard in production RAG systems. The reranker takes the top candidates from the PQ search (say, the top 100) and re-scores them using the original embeddings, recovering most of the recall loss at a fraction of the full-scan cost. We will cover this in detail in the chapter on reranking.

A second limitation is the static codebook. Once trained, the codebooks are fixed. If your data distribution shifts significantly, for example if you swap out your embedding model or index a new domain, the old codebooks become suboptimal and you need to retrain from scratch. Unlike HNSW graphs, which can be incrementally updated by inserting new nodes without retraining, PQ codebooks reflect a snapshot of your data distribution at training time. For systems that periodically replace their embedding model (which happens every time a better model is released), this means a full index rebuild, including codebook retraining and re-encoding all vectors. The rebuild process is well-parallelized and fast in practice, but it requires planning and downtime management.

Training cost is another consideration often overlooked in benchmarks. Running k-means with $K = 256$ over 50,000 training vectors in each of $M = 96$ subspaces is fast per subspace, but the overall training can take minutes on a CPU for large $M$. This is not prohibitive for most pipelines, but it means you cannot rebuild the index on every data update. Systems that require low-latency index updates (such as real-time document indexing) need to queue updates and batch-rebuild periodically, or maintain a separate uncompressed buffer for recently added vectors that is merged into the main PQ index at regular intervals.

The product structure assumption is a more fundamental limitation. PQ assumes that the dimensions of your embeddings are organized such that the correlations between dimensions mostly fall within subspaces rather than across them. In practice, most embedding models produce representations where correlations do span subspace boundaries. Optimized PQ (OPQ) addresses this by learning a rotation matrix that redistributes the variance so that more of the structure can be represented within the chosen subspaces. This improves recall at a given compression ratio, but adds the cost of applying the rotation at both encoding and query time. It is especially useful when the original dimension ordering splits strongly correlated features across subspace boundaries.

Finally, PQ distance estimates are always approximate. The error is bounded by the quantization quality, which depends on the total bit budget, the number of training samples, the intrinsic dimensionality of your data, and how well the data distribution matches the product structure assumed by independent per-subspace quantization. Datasets where nearby vectors share strong cross-subspace correlations tend to see lower recall for a given compression ratio. The heatmap in the previous section gives you a way to measure this: if you see high quantization error at fixed $K$, consider increasing $M$, increasing $K$, or applying OPQ. If the total code length must remain fixed, tune $M$ and nbits together rather than treating either parameter in isolation.

Despite these limitations, PQ remains one of the most practical tools in the retrieval engineer's toolkit. No other technique achieves comparable compression ratios with comparable retrieval quality. It reduces a previously intractable problem, fitting billions of high-dimensional vectors in RAM, to a manageable one by recognizing that distances decompose across subspaces. Every major production vector database in use today relies on some form of PQ or a closely related technique, and the original 2011 paper's core design choices remain essentially unchanged in modern implementations.