One-sentence Summary

Renmin University of China and Tencent Jarvis Lab propose MedRGAG, a unified retrieval-generation augmented framework for medical QA that integrates Knowledge-Guided Context Completion and Knowledge-Aware Document Selection to complement noisy retrieval with parametric generation, achieving a $12.5\%$12.5% improvement over MedRAG across five medical QA benchmarks.

Key Contributions

• MedRGAG is a unified retrieval-generation augmented framework for medical question answering that combines externally retrieved evidence and parametric knowledge generated by large language models to reduce reliance on noisy retrieval and hallucinated generation.

• Knowledge-Guided Context Completion uses knowledge gaps revealed by retrieval to guide generation of complementary background documents, while Knowledge-Aware Document Selection adaptively selects retrieved and generated documents to form concise evidence for answer generation.

• Experiments on five medical QA benchmarks and multiple reader architectures show consistent gains over strong baselines, including a 12.5% improvement over MedRAG and a 4.5% improvement over MedGENIE, and code and data are publicly released.

Introduction

Large language models can improve medical question answering, but their parametric knowledge is limited and can produce plausible yet factually incorrect answers, which is especially risky in clinical settings. Prior methods often rely on retrieval-augmented generation, where retrieved documents can be noisy or incomplete, or generation-augmented generation, where synthetic contexts may introduce hallucinations, and existing hybrids do not fully solve these complementary limitations. The authors propose MedRGAG, a unified retrieval and generation framework that combines external medical evidence with model-generated background knowledge, uses knowledge-guided context completion to fill missing information, and performs knowledge-aware document selection to build more reliable evidence for answer generation.

Dataset

• The authors use five widely used medical question answering benchmarks: MedQA, MedMCQA, MMLU-Med, PubMedQA*, and BioASQ-Y/N.
• All benchmarks are cast as multiple-choice QA tasks, with each question containing two to four candidate answers.
• The paper summarizes dataset statistics in Table 2 and provides additional details in Appendix B, but the provided text does not specify exact subset sizes, source splits, or filtering rules.
• The data is used for evaluation, with accuracy reported as the primary performance metric under standard evaluation practice.
• No training split, mixture ratio, cropping strategy, or metadata construction is described in the provided excerpt.

Method

MedRGAG is formulated as a multi evidence reader for medical multiple choice question answering. Given a question $q$q and an answer set $A = \{a_1, \ldots, a_{|A|}\}$A={a1​,…,a∣A∣​}, the base reader model $\mathcal{M}_r$Mr​ generates an answer from its internal parameters and a task specific prompt:

Because this formulation relies only on parametric knowledge, the method combines retrieval augmented generation and generation augmented generation. In retrieval augmented generation, a corpus $C = \{d_1, \ldots, d_{|C|}\}$C={d1​,…,d∣C∣​} and a retriever $\mathcal{R}$R provide external evidence:

and the reader predicts the answer as:

In generation augmented generation, a generator $\mathcal{M}_g$Mg​ produces tailored background documents:

and the reader then predicts the answer as:

The proposed framework unifies these two directions through three sequential stages. Refer to the framework diagram:

The first stage is source balanced evidence retrieval. Instead of merging all medical corpora into one large source and retrieving from the unified pool, the system treats multiple heterogeneous sources as separate corpora $\{C_1, C_2, \ldots, C_s\}${C1​,C2​,…,Cs​}. This prevents retrieval bias toward large or repeatedly represented information sources. For a given question $q$q, the retriever independently selects a fixed number of top documents from each corpus and unions them into an intermediate candidate set:

This balanced selection produces a more diverse candidate pool. The system then applies a reranking model, specifically MedCPT Reranker, which jointly encodes the question and each candidate document to assign fine grained relevance scores. The highest ranked documents are retained as the retrieved evidence set $D_r$Dr​.

The second stage is knowledge guided context completion. Its purpose is to repair gaps and noise in the retrieved evidence. Retrieved documents may cover only partial aspects of a medical question, and some may be irrelevant or misleading. The module processes the retrieved evidence through three reasoning steps. First, a summarizer extracts the useful knowledge from each retrieved document with respect to the question. If a document does not contain useful information, the model outputs a control signal indicating that no useful information was found. This step filters noise while preserving concise, question relevant summaries.

Second, an explorer model $\mathcal{M}_e$Me​ evaluates the summarized evidence and identifies important knowledge that is still missing for answering the question:

where $D_s$Ds​ denotes the summarized evidence and $\mathcal{K}$K is the set of missing knowledge points. Documents marked as having no useful information are excluded before exploration to avoid wasting reasoning capacity.

Third, a generator $\mathcal{M}_g$Mg​ produces background documents conditioned on each missing knowledge point:

This yields a set of complementary documents. If the number of missing knowledge points $m$m is smaller than the target number of generated documents $k$k, the system generates additional background documents directly from the question. This provides broader context and reduces overfitting to narrow knowledge fragments. The output of this stage is the generated evidence set $D_g$Dg​.

The third stage is knowledge aware document selection. Both retrieved and generated documents are merged into the candidate pool $D_r \cup D_g$Dr​∪Dg​. Simply concatenating all candidates would increase input length and may dilute useful information with redundant or noisy content. Therefore, an integrator model $\mathcal{M}_i$Mi​ performs knowledge aware selection. The integrator first identifies the essential knowledge components required by the question. It then maps each candidate document to one or more of these components, assigning irrelevant documents to a nonuseful group. Finally, it selects a compact subset of documents that collectively covers the required knowledge components while minimizing redundancy. Formally, the final evidence set is:

The reader model then receives the question together with this selected evidence and generates the final answer:

Experiment

The experiments evaluate MedRGAG on multiple medical multiple-choice question answering benchmarks and compare it with direct answering, retrieval-augmented, and generation-augmented baselines. The main results, ablations, and performance analyses show that combining retrieved evidence with generated medical knowledge is more effective than relying on either source alone, and that knowledge-guided context completion and adaptive document selection are important for assembling useful reasoning contexts. Additional analyses and a case study further indicate that stronger generator and auxiliary language models improve context quality, while adaptive selection balances retrieved and generated evidence to support more accurate medical reasoning.

Across the evaluated medical QA settings, MedR-GAG attains the highest average accuracy and consistently outperforms direct-response, retrieval-augmented, and generation-augmented baselines. Compared with standard retrieval-augmented approaches, it provides markedly better performance, indicating that generated knowledge can help cover missing information. Its adaptive document selection also balances retrieved and generated evidence while retaining accuracy with fewer documents. Adaptive document selection maintains strong accuracy while reducing document count and preserving useful retrieved evidence. Retrieved and generated documents are complementary, with generated passages supplying missing explanations and retrieved passages supporting factual details.

The evaluated medical question answering suite combines examination-based benchmarks and literature-based benchmarks, all expressed as multiple-choice tasks. Examination datasets generally have more answer options and longer question stems, while literature datasets are smaller and use shorter prompts with fewer options. Accuracy is used as the primary metric for assessing answer correctness. The benchmark mix covers both examination-style questions and literature-derived questions. Examination datasets are associated with longer question stems and mostly four candidate answers. Literature datasets use shorter prompts and fewer answer choices, including binary questions.

Ablation results show that the full system achieves the highest scores on all three medical QA datasets compared with direct response and each reduced variant. Removing the generation module causes the largest decline, suggesting generated documents provide especially useful question-specific knowledge, while removing retrieval also lowers accuracy and shows that external evidence remains important. Disabling the knowledge-guided completion or adaptive document selection component further reduces performance, confirming that these modules help construct and choose informative contexts. The full system outperforms direct response and all ablated variants on MedQA-US, MedMCQA, and MMLU-Med. Removing generation causes the largest relative accuracy drop among the tested ablations. Removing retrieval also degrades performance, showing that retrieved evidence contributes to reasoning. Removing knowledge-guided completion or adaptive selection reduces accuracy, supporting the value of targeted context completion and evidence filtering.

The case study shows that retrieved documents can provide partial evidence about an NF2 gene mutation but may omit key disease associations, while generated documents add missing reasoning context. The generated passages elaborate on NF2-related tumor risks and distinguish it from related syndromes, helping eliminate a distractor. Combining selected retrieved and generated evidence supports the correct answer, meningioma. Retrieved evidence identifies NF2 and tumor risk but provides incomplete explanation. Generated evidence enriches the reasoning by adding disease context and ruling out related syndromes. Adaptive selection combines retrieved and generated passages to reach the correct answer.

The experiments evaluate a medical question answering suite of examination-based and literature-based multiple-choice benchmarks using accuracy to compare MedR-GAG with direct-response, retrieval-augmented, and generation-augmented baselines, ablation variants, and a representative evidence case. Results show that MedR-GAG achieves the best overall performance, with ablations confirming that generated question-specific knowledge, retrieved factual evidence, knowledge-guided completion, and adaptive document selection all contribute meaningfully to reasoning quality. The case study further illustrates that retrieved and generated passages are complementary, as retrieved documents supply factual anchors while generated documents add missing explanation and help filter distractors.