AMD Unveils New Instinct MI355X GPU: A Leap in AI Inference Performance at 1,400W Power Consumption

At the International Supercomputing Conference (ISC) 2025, AMD’s Chief Technology Officer, Mark Papermaster, unveiled the new Instinct MI355X accelerators for artificial intelligence (AI) and high-performance computing (HPC). The MI355X, which is part of the Instinct MI350X series, is based on AMD’s CDNA 4 architecture. This architecture introduces support for FP4 and FP6 precision formats, in addition to FP8 and FP16, which are crucial for AI inference tasks. Key Features and Performance The Instinct MI355X is designed for enhanced AI inference performance. It features 288 GB of HBM3E memory, which offers up to 8 TB/s of bandwidth, a significant upgrade from its predecessor. In terms of performance, the MI355X delivers peak computational capabilities of 20.1 PFLOPS in FP4/FP6 precision and 10.1 PFLOPS in FP8 precision. This represents a substantial improvement over the previous model, the Instinct MI325X, which offered 5.22 PFLOPS in FP8 precision (without structured sparsity). The standard Instinct MI350X module consumes 1000 watts of power and is suited for air cooling. The higher-performance MI355X, however, has a peak power consumption of 1400 watts and is primarily designed for direct liquid cooling (DLC), although AMD acknowledges that some clients may find air cooling sufficient. Both models are expected to outperform Nvidia's Blackwell B300 Ultra GPU in FP4/FP6 and FP8 precision. Performance Comparison The performance gains of the MI355X are striking. Compared to the MI325X, it offers a 195% increase in FP8 performance and a 9.5% increase in FP6 performance. When stacked into a platform with eight OAM (Open Accelerator Module) units, the MI355X can achieve an impressive 40.27 PFLOPS in FP8 and 161.06 PFLOPS in FP4/FP6 precision. Industry Trends and Challenges Papermaster highlighted the rapid advancements and increasing power consumption of supercomputing accelerators. According to AMD, compute performance of top supercomputers has doubled approximately every 1.2 years since 1990. The transition from CPU-only systems to heterogeneous architectures, which combine CPUs with GPUs and accelerators, began around 2005. Today, systems like El Capitan and Frontier are breaking the exascale barrier, continuing this trend with specialized AI hardware. While performance is improving, power consumption is a major concern. AMD’s Instinct MI300X, introduced in 2023, had a peak power consumption of 750 watts. The MI355X, unveiled in 2025, will nearly double that to 1400 watts. AMD projects that by 2026-2027, accelerators could consume up to 1600 watts, and by the end of the decade, up to 2000 watts. To sustain performance growth, memory bandwidth must more than double every two years, necessitating the addition of more HBM stacks per GPU. This not only increases the physical size of the GPUs but also their power demands. Despite these challenges, there has been a rapid gain in performance efficiency, increasing from about 3.2 GFLOPS/W in 2010 to approximately 52 GFLOPS/W with exascale systems like Frontier. Future Outlook Looking ahead to the zettascale era, which is expected to deliver 1,000 times the performance of current exaflop-class systems, maintaining the performance growth rate will require a 41-fold increase in energy efficiency to approximately 2,140 GFLOPS/W. Without such gains, zettascale supercomputers could consume hundreds of megawatts of power, making them prohibitively expensive to operate. AMD is confident that achieving these performance milestones will require significant architectural breakthroughs and continuous innovation in memory bandwidth. Moreover, the industry may need to explore unconventional solutions for powering these systems, including the use of nuclear reactors, which could become a viable option by the 2030s. Industry Reactions and Company Profiles Industry insiders have praised AMD's latest accelerators for their performance improvements and innovative support for lower precision formats. The introduction of the Instinct MI355X is seen as a bold move in the AI and HPC space, positioning AMD to compete more effectively with market leaders like Nvidia. However, the near-doubling of power consumption has raised concerns about the sustainability and operational costs of these advanced systems. AMD, a leading semiconductor company, is known for its robust product lineup in both consumer and enterprise markets. The company’s focus on AI and HPC accelerators underscores its commitment to pushing the boundaries of computational technology. As one of the primary competitors to Nvidia in the GPU market, AMD’s advances in this domain are closely watched and often seen as a barometer for the industry’s overall progress.