IFEval-Inverse is an adversarial instruction evaluation dataset for large language models released in 2025 by ByteDance Seed in collaboration with Nanjing University, Tsinghua University and other institutions. The related paper results are "Inverse IFEval: Can LLMs Unlearn Stubborn Training Conventions to Follow Real Instructions?", which aims to test whether the model can break the training inertia and achieve true instruction following when faced with reverse or abnormal instructions. This dataset contains 1,012 high-quality bilingual Chinese and English question samples, covering eight types of unusual instruction challenges, including question correction, intentional text errors, uncommented code, unusual formatting, intentionally incorrect answers, leading questions, mid-course instruction revisions, and counterfactual question-answering, across 23 different domains. Each sample undergoes a combination of human-machine screening and validation, and utilizes the LLM-as-a-Judge automated scoring mechanism. This makes it suitable for evaluating and improving the adaptability and robustness of models in non-traditional instruction scenarios, and also provides an important benchmark for research on mitigating model cognitive inertia.

Citation

@misc{zhang2025inverseifevalllmsunlearn, title={Inverse IFEval: Can LLMs Unlearn Stubborn Training Conventions to Follow Real Instructions?}, author={Qinyan Zhang and Xinping Lei and Ruijie Miao and Yu Fu and Haojie Fan and Le Chang and Jiafan Hou and Dingling Zhang and Zhongfei Hou and Ziqiang Yang and Changxin Pu and Fei Hu and Jingkai Liu and Mengyun Liu and Yang Liu and Xiang Gao and Jiaheng Liu and Tong Yang and Zaiyuan Wang and Ge Zhang and Wenhao Huang} year={2025}, eprint={2509.04292}, archivePrefix={arXiv}, primaryClass={cs.CL}, url={https://arxiv.org/abs/2509.04292}, }