AetherCode is a programming competition evaluation dataset released by ByteDance and the MAP team in 2025. The related paper results are "AetherCode: Evaluating LLMs' Ability to Win In Premier Programming Competitions", which aims to more realistically evaluate the algorithmic reasoning and coding capabilities of large models through difficult questions from top competitions such as IOI, ICPC, USACO, and high-quality test cases verified by experts. This dataset is sourced from top global programming competitions and consists of two parts: v1_2024 (public set) with 400 problems and v1_2025 (private set) with 56 problems. The public set includes complete test cases and checkers, while the private set does not include test cases and is intended for blind evaluation. The questions cover ten categories: Basic, Search, Dynamic Programming (DP), Strings (Str.), Math, Data Structures (DS), Graphs (Graph), Geometry (Geo.), Technology (Tech.), and Trees. The dataset features authoritative questions, extensive coverage, and a high level of difficulty. The questions are formatted in Markdown+LaTeX, and test cases are automatically generated and reviewed by experts. It is suitable for scenarios such as code generation and algorithm reasoning evaluation, competition-level capability comparison, and model progress tracking.

Data difficulty distribution:

• Easy: 159 questions
• Medium: 145 questions
• Hard: 132 questions
• Extreme: 20 questions

Citation

@misc{aethercode,
title={AetherCode: Evaluating LLMs' Ability to Win In Premier Programming Competitions},
author={Zihan Wang and Jiaze Chen and Zhicheng Liu and Markus Mak and Yidi Du and Geonsik Moon and Luoqi Xu and Aaron Tua and Kunshuo Peng and Jiayi Lu and Mingfei Xia and Boqian Zou and Chenyang Ran and Guang Tian and Shoutai Zhu and Yeheng Duan and Zhenghui Kang and Zhenxing Lin and Shangshu Li and Qiang Luo and Qingshen Long and Zhiyong Chen and Yihan Xiao and Yurong Wu and Daoguang Zan and Yuyi Fu and Mingxuan Wang and Ming Ding},
year={2025},
eprint={2508.16402},
archivePrefix={arXiv},
primaryClass={cs.SE},
url={https://arxiv.org/abs/2508.16402},
}