CantTalkAboutThis Topic Control is a dataset released by NVIDIA in 2024 for training language models to maintain topic focus in task-oriented dialogues. The related paper is "CantTalkAboutThis: Aligning Language Models to Stay on Topic in Dialogues", aiming to improve model resistance to distractors and enhance alignment capabilities for instruction following and safety tasks.

The dataset contains 1,080 synthetic dialogues covering nine domains, including healthcare, banking, travel, education, finance, insurance, law, real estate, and computer troubleshooting. Each dialogue includes distractors to test and improve model performance when handling irrelevant topics. The dataset does not contain personal sensitive information and is generated from synthetic data.

Dataset Composition

The dataset contains the following fields:

• domain: The domain to which the dialogue belongs
• scenario: The specific scenario or task being discussed
• system_instruction: The dialogue strategy given to the model, typically a set of complex instructions regarding allowed and disallowed topics
• conversation: The complete dialogue, including the main topic and distractors
• distractors: A list of distractors, including the robot replies in the dialogue and the distractors the user should include in their responses to the robot replies
• conversation_with_distractors: The dialogue containing distractors

The dataset is divided into two subsets: the training set (mixtral) and the test set (human_test_set). The training set is generated based on the Mixtral-8x7B-Instruct model, and the test set contains more complex and realistic human-annotated distractors for evaluating model performance.

Citation

@inproceedings{sreedhar2024canttalkaboutthis,
  title={CantTalkAboutThis: Aligning Language Models to Stay on Topic in Dialogues},
  author={Sreedhar, Makesh and Rebedea, Traian and Ghosh, Shaona and Zeng, Jiaqi and Parisien, Christopher},
  booktitle={Findings of the Association for Computational Linguistics: EMNLP 2024},
  pages={12232--12252},
  year={2024},
  organization={Association for Computational Linguistics}
}