AetherCode is a programming competition evaluation dataset released by ByteDance and the MAP team in 2025. The related paper results are "AetherCode: Evaluating LLMs' Ability to Win In Premier Programming Competitions", which aims to more realistically evaluate the algorithmic reasoning and coding capabilities of large models through difficult questions from top competitions such as IOI, ICPC, USACO, and high-quality test cases verified by experts.

This dataset is sourced from top global programming competitions and consists of two parts: v1_2024 (public set) with 400 problems and v1_2025 (private set) with 56 problems. The public set includes complete test cases and checkers, while the private set does not include test cases and is intended for blind evaluation. The questions cover ten categories: Basic, Search, Dynamic Programming (DP), Strings (Str.), Math, Data Structures (DS), Graphs (Graph), Geometry (Geo.), Technology (Tech.), and Trees. The dataset features authoritative questions, extensive coverage, and a high level of difficulty. The questions are formatted in Markdown+LaTeX, and test cases are automatically generated and reviewed by experts. It is suitable for scenarios such as code generation and algorithm reasoning evaluation, competition-level capability comparison, and model progress tracking.

Data difficulty distribution:

• Easy: 159 questions

• Medium: 145 questions

• Hard: 132 questions

• Extreme: 20 questions