August 1,The 9th Meet AI Compiler Technical Salon successfully concluded in Beijing.Several experts and scholars in the field of AI compilers gathered in Zhongguancun, Beijing, to share their insights on technological innovation and practical experience, aiming to present a picture of the collaborative evolution of AI compilers across multiple levels, including language expression, operator computation, inference execution, and scenario applications.

Tianqi Chen, Assistant Professor in the Department of Machine Learning and the Department of Computer Science at Carnegie Mellon University, NVIDIA Distinguished Engineer, and a core developer and founder of globally renowned open-source projects XGBoost, Apache TVM, and MLC LLM.He attended the event as a special guest. This pioneering figure in the field of AI Compiler shared the current development direction of the TVM open source community and engaged in in-depth exchanges with the audience.

HyperAI has compiled and summarized Professor Chen Tianqi's sharing without altering the original meaning.

Hello everyone, I'm very happy to participate in this event and share my insights with you. I'm Chen Tianqi. We've been exploring the field of AI compilers for many years, and this year has been a very interesting one. Since the beginning of the year, the AI Agent wave has risen rapidly, and some new problems have begun to emerge in the industry:

The first question: Is there still value in AI compilers? When agents can be programmed directly, do we still need compilers?

The second question: With automation, we can do more things. In the past, the biggest bottleneck for small teams was engineering resources; with the arrival of agents, we can accomplish more with smaller teams.

So what I want to share today is:In the era of AI agents, how should we rethink AI compilers? What direction should future AI compilers take?

Accelerate Agent-Specific Iteration

A key characteristic is that agents require relatively rapid, dedicated iterations because they have limited context. In the past, we tended to favor large-scale project development, specifically large monolithic repositories, but now dedicated iterations for agents are becoming a very important direction.

First, in order to accelerate agent-specific iterations, we found that the stability, composability, and modularity of project boundaries are very important.

Taking our TVM FFI project as an example, the goal is to clearly define a core component within the large TVM framework—that is, how functions call each other. This includes defining the function boundaries between different components, from the framework DSL to the backend, such as cuteDSL and FlashInfer. In the past, we focused more on creating an end-to-end solution, but now we increasingly find that we need to define a more organic boundary that allows us to easily switch between operators or some modules. Therefore, clearly defining and standardizing the boundary itself becomes crucial.

Since its release last year, TVM FFI has been integrated with various technologies including cuteDSL, FlashInfer, and more recently, TensorRT. Its purpose is to define what the boundary specification should look like. Once the boundary is clearly defined, intermediate modules can be replaced more flexibly.

Secondly, to enable Agent to complete dedicated iterations quickly, you will see more cross-language support in the future.

If the agent's feedback time is one or two hours, then its trial and error time will become longer and longer. Therefore, the compiler itself needs to support more languages, so we will push for infrastructure-level support for more languages.

In the past, users typically interacted with a program, which itself was the object of their interaction. However, because AI can be more intelligent, we may find that people will not directly perform all calculations, but will create tools. For example, when doing mathematical calculations, you don't necessarily have to do it mentally; you can use tools instead.Similarly, we believe that the compiler's internal analytical capabilities and infrastructure will gradually come to the forefront. We hope to modularize the compiler, allowing AI to build its own analysis and iteration processes tailored to current needs.

To achieve this, we've recently been working on TIRx, and we've also released a tutorial called Modern GPU Programming. Of course, it's not limited to GPUs; it attempts to introduce more general accelerator programming concepts. In the last six months, I mentioned to Feng Siyuan that I've spent about half my time writing code. Because with the Agent, we can indeed go back and refactor the entire project's code. Last year, we made the TVM FFI module relatively independent and stable, and we will further promote the independence and stabilization of TIRx.

In the Agent era, what can compilers do?

We have found that traditional compilers have a significant characteristic: they typically put two types of things in the same stack. One type is the high-level, intelligent part, such as automatic scheduling; the other type is the low-level, less intelligent part, such as purely rule-based things.However, we believe the trend will change in the future: we increasingly need to separate the stable parts from the parts that require rapid iteration. In other words, there will be a core module that is relatively stable; on this basis, we can more flexibly add many agent-related capabilities, allowing agents to build dedicated compilers and optimize processes according to their needs.

Returning to the main topic: In the Agent era, what can compilers do?

I think that by this time next year, or even by the end of this year, the AI compilers we discuss will definitely be different from those we discussed before this year. This is a very interesting opportunity for us to rethink this together.

Conversely, humans with tools are certainly stronger than humans with only physical bodies. AI compilers are themselves a type of tool. Therefore, as long as we have the ability to better involve AI in the design and development of compiler-related modules, there will still be many interesting opportunities.

That's all I wanted to share, thank you everyone.

Q&A Session

Question 1: With the emergence of more and more hardware, should future compilers adopt general-purpose architecture adaptation or hardware-specific adaptation?

Chen Tianqi:With the advent of agents, many things will change. Traditionally, when we can share code across different backends, everyone will naturally prefer to share code. This is why we initially tended to build larger monorepo systems for unified development. However, with agents, the cost of developing code will decrease, so more specialized parts will definitely emerge in the future. This is especially true when the abstraction of shared code is insufficient to support the capabilities of the hardware itself; this often happens. For example, even on GPUs, a compiler designed for Hopper might become completely unusable after Blackwell. This occurs frequently, so specialization will be essential in the future.

Conversely, the question remains: what can remain common? We still need a relatively stable foundation upon which agents can build hardware-specific compilers. For example, our recent work on TIRx aims to allow different hardware to register their own dispatch rules, and then let the agent generate relevant knowledge based on the hardware characteristics.

In the future, we will see more and more compilers that can be specialized. It will likely be divided into two parts—the first part is the more stable part, such as ABI requirements, core IR data structures, and cross-language capabilities, which have high stability requirements; the second part is the part specialized for specific scenarios, which will likely be more agent-driven in the future. I think we will see more and more of this in the future.

Question 2: During my learning process, I've noticed that many traditional abstract boundaries in IR are being broken down and merged. Things that were previously clearly defined seem to be gradually being absorbed by IR. From this perspective, what do you think is the future development direction of AI compilers?

Chen Tianqi:I think it will be divided into two parts. First, although different IRs may look different now, they are actually quite similar in many ways, and can basically be divided into several levels: kernel level, graph level, and the interaction between them. After a few years of evolution, it may gradually converge to a few more stable levels. Too many layers are not actually beneficial, because information is often lost when transitioning between layers. So once we converge to a few main levels, the core issue may become: how do we encode the necessary information?

The second part addresses the question of whether many tasks previously requiring a strong compiler can be delegated to external personnel or agents for control. In other words, can the compiler be made thinner, or layered? Layering means having one relatively stable layer that can be continuously maintained, and another layer that can iterate rapidly—at least in the future, these two layers will definitely exist.

Question 3: From the perspective of AI compilers, will we see AI writing AI compilers in the future? In the short term, which compiler components are suitable for AI to write? What are the main challenges involved?

Chen Tianqi:First of all, I believe that many practitioners here have already started using AI to write compilers, so this is not a question of future possibility, but something that is happening now.

Modularization is crucial here. It's certainly difficult for AI to fully manage a large project; however, it's entirely possible to delegate certain modules to AI for rapid iteration and trial and error. The key is to break the problem down into well-defined subproblems. Therefore, boundary definition is essential. Our TVM FFI approach aims to clearly define these boundaries.

Similarly, a good IR infrastructure is also crucial. For example, currently, if you want to write a Compiler IR that supports languages such as Python, Rust, and C++, there isn't a perfectly feasible solution. In the future, if we can provide such infrastructure, it will greatly improve development efficiency.

So what can we do beyond infrastructure? I think it depends on whether we can extract some common needs. For example, we've discussed a problem before: a significant part of the compiler is host-side cost reduction. This problem is actually relatively independent of GPU acceleration and hardware acceleration. However, there's still a lot of room for automation in host-side optimization itself. If we can extract these common needs, AI will play a greater role in compiler development in the future.

Question 4: What positions will different levels of Compilers, such as low-level DSL and high-level DSL, occupy in the industry and development ecosystem in the future?

Chen Tianqi: I think different levels will have their own needs. Generally speaking, it's good to be able to expose the Low level while also having representations for both Low and High levels. For example, TIRx, which we recently developed, has its own representations for different levels, as well as representations for the Native partner level, allowing for flexible transitions between the two. However, there's a principle: the Agent should use as few tokens as possible to represent these things, while not limiting its capabilities.

Question 5: In the past, when we discussed compilers, we focused more on the kernel level, but in reality, many things can be represented using IR. So, in the future, besides kernel-level IR, will micro-level IR, and even scheduling-level IR, become research directions for AI compilers?

Chen Tianqi:This is actually quite normal, because graph-level compilers have always existed. It's just that in the past, the interaction between the kernel and graph-level compilers was relatively weak. Graph-level compilers mainly solve graph-level problems, while kernel-level compilers mainly solve kernel-level problems. However, in the future, we will increasingly need interaction between the two.

A major recent research direction is MegaKernel, which requires cross-kernel interaction. Therefore, we've been considering how to encapsulate both types of information within a single IR (Integrated Resource Identifier). If this can be achieved in the future, I think it will lead to some interesting developments. For example, previous work on MegaKernel essentially utilized the interaction between the graph level and the kernel level to achieve optimization.

Question 6: Is the Agentic-friendly Compiler a true proposition?

Chen Tianqi:A future scenario might emerge where we don't need a Compiler; the Agent can directly generate the Assembly. The difference lies in this: we can have the Agent perform this task directly, or we can have the Agent write the Compiler, creating the tool itself. Personally, I still believe the Compiler has value because it can improve development efficiency in many situations.

So, is "agent-friendly" a false proposition or a true one? I think the key may not lie in whether your software initially possesses this characteristic, but rather in whether we design it as a goal. Once the agent is involved, we continuously observe: where the agent performs well, where it performs poorly, and then iterate based on this feedback.

For example, cross-language requirements are a very real problem. An agent might initially be able to quickly develop a Compiler Pass using Python, but when it needs acceleration, it might need to switch to C++ or Rust. If we take these problems as our goals, I believe it won't be a bad thing, because a significant proportion of code is already written by agents. In the future, the Compiler will also become a tool that agents can use to help build and optimize.

The above is a summary of the insightful presentation by Professor Chen Tianqi at the Meet AI Compiler Technical Salon. Other exciting speeches from this event will also be reported in the form of transcripts, so stay tuned!