The CLaRa (Continuous Latent Reasoning) dataset is a collection released by Apple in 2025 for retrieval-augmented generation and continuous latent reasoning, with related research findings presented in the paper titled "CLaRa: Bridging Retrieval and Generation with Continuous Latent Reasoning", aiming to provide training and evaluation data to support model performance in Question Answering under compressed document representations.

This dataset includes three major categories of data: pre-training, instruction fine-tuning, and end-to-end tuning. It covers construction and evaluation sets for various mainstream QA benchmarks, such as 2WikiMultihopQA, HotpotQA, MuSiQue, and Natural Questions. The dataset is large-scale (ranging from 10 GB to 100 GB), uniformly formatted in JSONL, supports both positive-document scenarios (Oracle) and normal retrieval settings, making it suitable for research on Retrieval-Augmented Generation (RAG), document compression, and complex QA tasks.

Dataset Composition

The dataset primarily consists of the following three parts, with all data stored in JSONL format:

• Pretraining Data: Pre-training data used for learning the document compressor. Contains fields:

  • data_type: Data type (e.g., qa)
  • question: List of questions
  • answers: List of answers
  • docs: List of document contexts

• Fine-Tuning Data: Instruction fine-tuning data used for QA training based on compressed document representations. Contains fields:

  • question: Question string
  • docs: List of retrieved documents
  • gold_answer: Ground truth answer
  • answer: Model-generated answer

Additionally, there are evaluation sets for different benchmarks (2wiki, hotpotqa, musique, nq), divided into two setups: those including positive documents (Oracle setup: inst_eval*_with_pos) and standard setups without them (inst_eval*_no_pos).

• End-to-End Tuning Data: Complete workflow data used for end-to-end model training. Contains fields:
  • question: Question string
  • answer: Answer string
  • docs: List of retrieved documents
  • pos_index: Index of the positive document within the docs list

These are also split into training and evaluation sets, providing two configurations—those containing positive examples (with_pos) and those excluding them (no_pos)—for different benchmark tests (ours_2wiki, ours_hotpotqa, ours_musique, ours_nq).

End-to-End Tuning Data: Complete workflow data used for end-to-end model training. Contains fields:

• question: Question string
• answer: Answer string
• docs: List of retrieved documents
• pos_index: Index of the positive document within the docs list

Similarly divided into training and evaluation sets, this category offers two configurations—with positive samples (with_pos) and without (no_pos)—across distinct benchmark evaluations (ours_2wiki, ours_hotpotqa, ours_musique, ours_nq).

Citation```bibtex

@misc{he2025clarabridgingretrievalgeneration, title={CLaRa: Bridging Retrieval and Generation with Continuous Latent Reasoning}, author={Jie He and Richard He Bai and Sinead Williamson and Jeff Z. Pan and Navdeep Jaitly and Yizhe Zhang}, year={2025}, eprint={2511.18659}, archivePrefix={arXiv}, primaryClass={cs.CL}, url={https://arxiv.org/abs/2511.18659}, }