Paper2Code: Automating Code Generation from Scientific Papers in Machine Learning

Despite the rapid growth of machine learning research, corresponding codeimplementations are often unavailable, making it slow and labor-intensive forresearchers to reproduce results and build upon prior work. In the meantime,recent Large Language Models (LLMs) excel at understanding scientific documentsand generating high-quality code. Inspired by this, we introduce PaperCoder, amulti-agent LLM framework that transforms machine learning papers intofunctional code repositories. PaperCoder operates in three stages: planning,where it constructs a high-level roadmap, designs the system architecture withdiagrams, identifies file dependencies, and generates configuration files;analysis, which focuses on interpreting implementation-specific details; andgeneration, where modular, dependency-aware code is produced. Moreover, eachphase is instantiated through a set of specialized agents designed tocollaborate effectively across the pipeline. We then evaluate PaperCoder ongenerating code implementations from machine learning papers based on bothmodel-based and human evaluations, specifically from the original paperauthors, with author-released repositories as ground truth if available. Ourresults demonstrate the effectiveness of PaperCoder in creating high-quality,faithful implementations. Furthermore, it consistently shows strengths in therecently released PaperBench benchmark, surpassing strong baselines bysubstantial margins.