Free-Threaded Python: First Year Milestones and the Path Forward

Last week, the CPython developers released CPython 3.14.0b1, marking a significant milestone in the ongoing effort to deliver and stabilize free-threaded Python. This week, PyCon 2025 is underway in Pittsburgh, PA, further highlighting the importance of this initiative. Over the past year, the development team, including key contributors from Quansight, has been instrumental in enabling the experimental use of the free-threaded build in real production workflows. Why Free-Threaded Python Matters Free-threaded Python aims to unlock the full potential of modern hardware, which typically comes equipped with multicore CPUs and GPUs. In the current Global Interpreter Lock (GIL)-enabled build, achieving efficient parallel computing in Python requires workarounds and careful tuning. For example, the threading module’s effectiveness is hampered by the GIL, which serializes access to Python objects, preventing useful parallel scaling. Developers often opt for the multiprocessing module instead, but this approach is resource-intensive and involves costly data copying across processes. Milestones and Accomplishments Quansight, in collaboration with the Python runtime team at Meta, has made substantial contributions to the free-threaded Python effort. These include enhancing packaging and project workflow tools, bindings generators, and foundational packages within the PyData ecosystem. Notable projects that now support the free-threaded build are: Packaging Tools: meson, meson-python, setup-python GitHub Actions workflow, packaging, pip, and setuptools. Bindings Generators: Cython, pybind11, f2py, and PyO3. PyData Ecosystem: NumPy, SciPy, PyArrow, Matplotlib, pandas, scikit-learn, and scikit-image. High-Demand Packages: Pillow, PyYAML, yarl, multidict, and frozenlist. These improvements collectively pave the way for more efficient and scalable Python applications. Additionally, Quansight has developed a comprehensive guide to help package maintainers and developers transition their apps and packages to the free-threaded build. The guide addresses common issues and provides best practices for ensuring thread safety and performance. Current State of the Ecosystem A year ago, when Python 3.13.0b1 was released, the broader Python package ecosystem was largely incompatible with the free-threaded build. Simple packages without dependencies could be installed, but more complex ones often resulted in build errors. Since then, significant progress has been made in addressing these issues. The release of Cython 3.1.0, which officially supports the free-threaded build, has resolved one of the major sources of build problems. Quansight is actively working on bringing support for the free-threaded build to popular packages that still lack it, such as CFFI, cryptography, PyNaCl, aiohttp, SQLAlchemy, grpcio, safetensors, and tokenizers. You can monitor the progress of these efforts through Quansight’s manually updated status tracking table or Hugo van Kemenade’s automated tracker. Challenges Remaining While the free-threaded Python build is now viable for experimentation, several challenges persist. The primary issues include: Performance Bugs: Real-world workflows are essential for identifying performance bottlenecks and unexpected bugs. Developers are encouraged to test their applications and report any anomalies. Thread Safety Audits: Many packages that ship compiled code still require detailed audits to ensure they can handle simultaneous access without data races. Libraries with mutable data structures are especially problematic in this regard, and often lack documentation on thread safety or multithreaded performance. Resource Constraints: Popular and legacy packages may lack the necessary resources to address the changes required for free-threading. Community support is crucial to ensure sustainable maintenance of these critical components. How You Can Help Quansight has provided a contribution guide in their free-threading documentation, outlining ways to assist in the ecosystem-wide effort. Contributions are tracked in the free-threaded-compatibility repository on the Quansight-Labs GitHub organization. Additionally, a community Discord server facilitates discussions and collaborations around supporting the free-threaded build. If you’re attending PyCon 2025, consider joining the talk by Quansight’s Lysandros Nikolaou and the author. They will share insights and lessons learned from porting packages to the free-threaded build, offering valuable guidance for both attendees and online viewers. Industry Evaluation and Company Profile Industry insiders view the free-threaded Python build as a pivotal step toward leveraging modern hardware effectively. Quansight, known for its contributions to open-source software and scientific computing, has positioned itself at the forefront of this effort. By fostering a community-driven approach and providing resources such as comprehensive guides and tracking tools, Quansight is facilitating a smoother transition to this new, more powerful build of Python. The potential benefits of free-threaded Python are profound. It promises to significantly enhance the performance of data-intensive and parallel computing workflows, reducing the overhead associated with process spawning and data copying. Furthermore, it aligns with the increasing trend of multicore and GPU-based architectures in the tech industry, making Python a more viable option for high-performance computing. In conclusion, the first year of free-threaded Python has set a strong foundation, but the journey is far from over. Continued community involvement and support are essential to realize its full potential and transform the Python ecosystem.