Command Palette
Search for a command to run...
Event Recap | From Compilation Optimization to Scenario Applications: A Multi-Perspective Analysis of AI Compiler Technology Innovation and Implementation Experience in the Agent Era
On August 1st, the 9th Meet AI Compiler Technical Salon successfully concluded in Zhongguancun, Beijing! This session featured Chen Tianqi, a pioneer in the AI Compiler field, along with several technical experts from leading technology companies. They shared their insights on programming languages, operator optimization, inference execution, and application scenarios, areas in which they have long been deeply involved. From the continuous evolution of the open-source community to the practical application of compiler optimization in real-world scenarios, different technical paths converged here, making the technical landscape of AI compilers more comprehensive.
Follow the WeChat official account "HyperAI" and reply with the keyword "0801 AI Compiler" to get the speaker's complete PPT.
This time, we saw many familiar faces and made many new friends. On stage were technical practices from different teams, while off stage, developers were asking questions and exchanging ideas about technical details. From a sharing session to a Q&A session, and then to discussions that continued during the tea break, many thoughts were extended in new ways through this collision. Perhaps this is why we have always insisted on creating Meet AI Compiler—to allow people who are working on AI compilers to meet each other, to give opportunities for exploration in different directions to collide, and to create a truly connected and warm technical community for a group of people who have come together because of technology.
Event content review
Share Review
Share topic:In the era of AI agents, how should we rethink AI compilers? What direction should future AI compilers take?
Click to view the full sharing record:
Chen Tianqi: In the Agent Era, How Should We Rethink AI Compilers?
Share topic:FlagTree: Triton-TLE Language Extensions, Tile IR Backend, and Compiler Optimization Practices
Contents:This presentation will be divided into three parts. The first part introduces the challenges faced by Triton and how TLE (Tracked Operators) progressively exposes hardware details through three levels of language extensions, achieving a better balance between portability, maintainability, and performance. The second part focuses on the engineering practice of integrating Tile IR into the Triton compiler FlagTree, introducing how it, as a new type of compiler backend, further expands the performance optimization space of Triton operators. The third part will systematically analyze key compiler optimization techniques such as layout optimization and instruction reordering, demonstrating a complete compilation optimization path for cross-chip high-performance operators.
Watch this sharing session and you will learn:
1. How does TLE control on-chip memory, how does it express distributed and producer-consumer models, and how does it inline the vendor's native language?
2. How can TLE and Tile IR backends improve the performance ceiling of Triton key operators?
3. How compiler optimization techniques reduce data layout transformation overhead, improve instruction execution efficiency, and further unleash the performance of the Triton operator.
Click to view the full sharing record:
Share topic:TileRT: Speed is Intelligence – Computational Exploration and Co-design for Ultra-Low Latency Large Model Inference
Contents:As large models reach trillions of parameters and enter the agentic era, extreme inference speed has become a key element in supporting complex task flows and fully unleashing the model's potential. However, when systems attempt to further push latency limits, traditional system architectures and execution bottlenecks often become insurmountable obstacles.
This report introduces the latest explorations of TileRT, demonstrating how to build a software stack for large-scale model computing with ultra-low latency, from the perspective of AI compilers, runtime architecture evolution to model-system co-design.
Watch this sharing session and you will learn:
1. Speed is Intelligence: Exploring why "speed" is gradually becoming a key indicator for the inference side of large models in the Agentic era.
2. System Architecture Exploration: This section introduces the architectural evolution of TileRT, using GLM-5 as an example to discuss how to significantly improve inference performance by refactoring the underlying computation scheduling.
3. Model-System Co-design and Production Practice: Sharing how to break through the 1000 TPS speed bottleneck in single-batch inference for trillion-parameter models through joint design of models and systems.
Click to view the full sharing record:
Share topic:FalconGEMM: Surpassing Hardware Peaks with Lower-Complexity Matrix Multiplication
Contents:Matrix multiplication (GEMM) is the core of computational power for training and inference of large models. However, as model size grows exponentially, the O(N³) complexity of the standard algorithm is constantly approaching the physical peak of the hardware. How to continue to extract performance when chip computing power is at its peak has become a key issue for large model infrastructure. Understanding the principles, value, and engineering challenges of low-complexity matrix multiplication breaking through the performance ceiling in the context of operator optimization reaching its peak is crucial. One path explored by the mathematical community for more than 50 years is low-complexity matrix multiplication (LCMA, such as Strassen and AlphaTensor) – exchanging fewer multiplications for more memory accesses and additions, thus "breaking through" the hardware peak in an equivalent sense. However, the three major engineering challenges of memory access bloat, algorithm selection, and cross-platform portability have kept it at the theoretical level for a long time.
This report introduces the FalconGEMM project, which systematically brings LCMA from paper to the production-grade software stack, covering three levels: compiler-automated code generation, memory access optimization through group parallel fusion, and algorithm decision-making based on performance models. It also achieves a comprehensive surpassing of top-level official libraries on various GPU/CPU platforms and real large model workloads.
Watch this sharing session and you will learn:
1. Understand the principles, value, and engineering challenges of low-complexity matrix multiplication breaking through the performance ceiling when operator optimization reaches its limit.
- FalconGEMM's technical solutions and cross-platform practices.
Click to view the full sharing record:
Share topic:AscendNPU IR: The compilation platform is open source and supports multi-language integration with Ascend.
Contents:AscendNPU IR, the Ascend compiler component, has been fully open-sourced. As the MLIR access layer for Ascend to third-party programming frameworks, it provides flexible integration, complete expression, and Ascend-friendly compilation optimization capabilities, and supports multiple front-end DSLs to improve the performance of Ascend operators.
Watch this sharing session and you will learn:
1. Understand the overall technical architecture and design philosophy of AscendNPU IR.
2. Understand the new features of Ascend NPU IR for Ascend 950 expansion.
3. Understand the AscendNPU IR community building activities and how to participate.
Click to view the full sharing record:
Share topic:A general-purpose AI compiler for the field of embodied intelligence
Contents:This report introduces a general-purpose compiler for embodied intelligence and multimodal large models, focusing on the capture, export, grouping, compilation, runtime deployment, and performance optimization of complete algorithm pipelines, addressing key issues in edge delivery, stable operation, cross-framework adaptation, and engineering scaling of robot models.
Watch this sharing session and you will learn:
1. Understand the core challenges that distinguish embodied intelligence model deployment from traditional model deployment, including the engineering complexity and maintenance costs brought about by multiple models, multiple frameworks, and multi-stage pipelines.
2. Master how a general-purpose compiler can capture the complete algorithm flow through dynamic tracing, and organize modules such as preprocessing, VLA model, LLM, and post-processing into a compilable, deployable, and deployable DAG template.
3. Understand how grouped compilation and a unified runtime architecture support different backends, leveraging the advantages of various chips while maintaining a unified delivery chain.
4. Understand the interface paradigm between the embodied domain compiler and the distribution platform.
Click to view the full sharing record:
Organizers and partners
HyperAI (hyper.ai) is an internationally leading artificial intelligence and high-performance computing community.It aims to help developers and enthusiasts in the global data science and artificial intelligence industry learn, understand and practice by providing a series of services such as industry information reports, accelerated data set downloads, online tutorial demonstrations, popular model performance evaluations, cutting-edge paper recommendations, high-value results interpretations, and top conference calendar integration, and build the future of artificial intelligence together with the community.
Visit the official website:https://hyper.ai/
OpenBayes Bayesian Computing is a leading high-performance computing service provider in ChinaBy grafting classic software ecosystems and machine learning models onto new-generation heterogeneous chips, it provides industrial enterprises and university scientific research with faster and easier-to-use data science computing products. Its products have been adopted by dozens of large industrial scenarios or leading scientific research institutes.
Visit the official website:https://openbayes.com/
The MLC.AI community was established in June 2022. Chen Tianqi, the main inventor of Apache TVM and a well-known young scholar in the field of machine learning, led the team to launch the MLC online course, which systematically introduced the key elements and core concepts of machine learning compilation.
In November 2022, with the joint efforts of MLC.AI community volunteers, the first complete TVM Chinese documentation was launched and successfully hosted on the HyperAI official website, further providing domestic developers interested in machine learning compilation with the basic settings for accessing and learning a new technology - documentation.
MLC Online Courses:https://mlc.ai/
TVM Chinese Documentation:https://tvm.hyper.ai/
Event venue support
Beijing Zhongguancun Science City Innovation Development Co., Ltd., established in May 2019, is positioned as an ecosystem-driven technology investment and industrial service platform serving the construction of world-leading science and technology parks. In line with Haidian District's strategic deployment to build a "1+X+1" modern industrial system, it focuses on five core business segments: industrial services, ecological cooperation, technology investment, space operation, and international business. It constructs a comprehensive, multi-faceted service system covering the entire lifecycle of technology companies, promoting the deep integration of technological innovation and industrial development in Haidian District, striving to create a high ground for new-quality productivity development, and contributing to the construction of world-leading science and technology parks.
Event Support
Scan the QR code and remark "AI Compiler" to join the event group
