IFEval-Inverse Reverse Instruction Evaluation Dataset

IFEval-Inverse is an adversarial instruction evaluation dataset for large language models released in 2025 by ByteDance Seed in collaboration with Nanjing University, Tsinghua University and other institutions. The related paper results are "Inverse IFEval: Can LLMs Unlearn Stubborn Training Conventions to Follow Real Instructions?", which aims to test whether the model can break the training inertia and achieve true instruction following when faced with reverse or abnormal instructions.

This dataset contains 1,012 high-quality bilingual Chinese and English question samples, covering eight types of unusual instruction challenges, including question correction, intentional text errors, uncommented code, unusual formatting, intentionally incorrect answers, leading questions, mid-course instruction revisions, and counterfactual question-answering, across 23 different domains. Each sample undergoes a combination of human-machine screening and validation, and utilizes the LLM-as-a-Judge automated scoring mechanism. This makes it suitable for evaluating and improving the adaptability and robustness of models in non-traditional instruction scenarios, and also provides an important benchmark for research on mitigating model cognitive inertia.