Dense Passage Retrieval & RAG (2020) › Limitations and Challenges

Despite their significant advantages, DPR and RAG faced several important limitations that would be addressed in subsequent research. Understanding these limitations helps explain both why these systems marked an important milestone and why research continued to evolve beyond these initial approaches.

DPR's effectiveness depended on the quality and coverage of its knowledge base. If relevant information was absent or a topic was poorly represented, the system could not generate an accurate answer about it. Effective RAG systems therefore required knowledge bases with broad topic coverage and reliable sources.

The retrieval process itself could fail in several ways. If the query encoder did not properly understand the question's intent, it might retrieve irrelevant passages. If the passage encoder did not capture the semantic content of passages accurately, semantically relevant passages might be missed. These retrieval failures would propagate to the generation stage, leading to incorrect or irrelevant generated text.

While DPR enabled efficient retrieval from large knowledge bases using approximate nearest neighbor search, the overall RAG system remained computationally expensive. Retrieving passages required encoding the query, searching the knowledge base, and then encoding retrieved passages. Generating text required running the language model with extended context windows that included retrieved passages. For real-time applications with strict latency requirements, this computational overhead could be problematic.

The need to maintain and update knowledge bases also created ongoing computational and storage costs. Large knowledge bases required significant storage capacity, and updating them required re-indexing passages, which could be computationally expensive. These operational requirements limited RAG's applicability to resource-constrained environments.

Language models had limited context windows, constraining the number of retrieved passages that could be included in the generation context. If a question required information from many sources or from several long passages, the system might not fit all necessary context. The resulting answer could be incomplete or fragmented.

The sequential generation process in RAG-Token mode, where different passages could be used for different tokens, added computational complexity. Managing multiple retrieved passages and determining which to use for each generation step required additional processing, making this mode more expensive than RAG-Sequence while not always providing proportional benefits.

While DPR required less training data than previous neural retrieval methods, it still needed question-answer pairs with corresponding passages for effective training. Creating these datasets remained labor-intensive, particularly for specialized domains or languages with limited resources. The quality of training data also significantly influenced system performance, requiring careful curation and annotation.

RAG's joint training of retriever and generator required examples linking questions to relevant passages and target answers. Creating enough high-quality examples across specialized domains remained difficult, limiting the system to areas with sufficient training resources.

While RAG reduced hallucination compared to standard language models by conditioning generation on retrieved evidence, the system could still produce incorrect information. The generator might misinterpret retrieved passages, combine information from multiple passages incorrectly, or generate text that went beyond what was supported by the retrieved evidence. This limitation highlighted the ongoing challenge of ensuring factual accuracy in generated text, even when systems had access to relevant source material.

The system's reliance on retrieved passages meant that biases or inaccuracies present in the knowledge base could propagate to generated outputs. If the knowledge base contained biased information, outdated facts, or incomplete coverage of certain topics, these issues would affect RAG's outputs. Ensuring knowledge base quality remained an important concern for deployment in production systems.