Dataset Overview

ChatQA 2 Long SFT Data is a dataset released by NVIDIA in 2024 that focuses on long-context conversational question answering and Retrieval-Augmented Generation (RAG) capabilities. The related paper can be found at 「ChatQA 2: Bridging the Gap to Proprietary LLMs in Long Context and RAG Capabilities, aiming to improve the performance of large language models in handling tasks involving long contexts and complex multi-turn conversations.

The long_sft subset is derived from samples generated using GPT-4 from LongAlpaca12k, Open Orca, and Long Data Collections; the NarrativeQA_131072 subset was synthesized based on NarrativeQA by adding paragraphs relevant to given ground-truth summaries. This dataset serves as the full long-context Supervised Fine-Tuning (SFT) training data for the ChatQA-2 model and primarily consists of two subsets: long_sft and NarrativeQA_131072. The data format is JSON, containing both training and test sets used during the instruction fine-tuning phase of the model.

Dataset Composition

The dataset includes the following two main configurations (subsets):

long_sft

Long-context supervised fine-tuning data sourced from existing datasets, designed to enhance the model's understanding and generation abilities in long-text dialogues.

• Training set: long_sft/long_sft_QA_train.json
• Test set: long_sft/long_sft_QA_dev.json

NarrativeQA_131072

Synthesized long-context Q&A data based on the NarrativeQA dataset, enhanced with additional context-relevant paragraphs.

• Training set: NarrativeQA_131072/NarrativeQA_131072_QA_train.json
• Test set: NarrativeQA_131072/NarrativeQA_131072_QA_dev.json

Citation

@article{xu2024chatqa,
  title={ChatQA 2: Bridging the Gap to Proprietary LLMs in Long Context and RAG Capabilities},
  author={Xu, Peng and Ping, Wei and Wu, Xianchao and Liu, Zihan and Shoeybi, Mohammad and Catanzaro, Bryan},
  journal={arXiv preprint arXiv:2407.14482},
  year={2024}
}