ChatQA 2 Long SFT Data is a dataset released by NVIDIA in 2024, focusing on long-context conversational question answering and retrieval-augmented generation (RAG) capabilities. The related research paper is "ChatQA 2: Bridging the Gap to Proprietary LLMs in Long Context and RAG Capabilities", aimed at improving the performance of large language models in handling long-context and complex multi-turn dialogue tasks.

Among them, the long_sft subset is derived from GPT-4 samples in LongAlpaca12k, Open Orca, and Long Data Collections; the NarrativeQA_131072 subset is synthesized based on NarrativeQA by adding paragraphs related to given reference summaries. This dataset serves as the full long-context supervised fine-tuning (SFT) training data for the ChatQA-2 model and mainly includes two subsets: long_sft and NarrativeQA_131072. The data format is JSON, containing training and test sets, used for the model's instruction fine-tuning stage.

Dataset Composition

The dataset includes the following two main configurations (subsets):

• long_sft: Long-context supervised fine-tuning data derived from existing datasets, used to enhance the model's understanding and generation capabilities in long-text dialogues.

  • Training set: long_sft/long_sft_QA_train.json
  • Test set: long_sft/long_sft_QA_dev.json

• NarrativeQA_131072: Long-context question answering data synthesized based on the NarrativeQA dataset, expanding contextual information by adding relevant paragraphs.

  • Training set: NarrativeQA_131072/NarrativeQA_131072_QA_train.json
  • Test set: NarrativeQA_131072/NarrativeQA_131072_QA_dev.json

Citation

@article{xu2024chatqa,
  title={ChatQA 2: Bridging the Gap to Proprietary LLMs in Long Context and RAG Capabilities},
  author={Xu, Peng and Ping, Wei and Wu, Xianchao and Liu, Zihan and Shoeybi, Mohammad and Catanzaro, Bryan},
  journal={arXiv preprint arXiv:2407.14482},
  year={2024}
}