AMD Unveils MI350X and MI355X AI GPUs: Up to 4X Faster, 35X Inference Boost, 1,400W Power Draw

AMD recently unveiled its new MI350X and MI355X GPUs for AI workloads at the Advancing AI 2025 event in San Jose, California. These accelerators represent a significant step forward in performance, aiming to close the gap with market leader Nvidia. According to AMD, the new GPUs offer a 3X performance boost over the previous-generation MI300X, positioning the company to be more competitive in the AI and high-performance computing (HPC) markets. Key Features and Specifications Both the MI350X and MI355X are built on the CDNA 4 architecture and feature 288GB of HBM3E memory, with 8 TB/s of memory bandwidth. The main difference lies in their power consumption and cooling methods: - MI350X: Designed for air-cooled solutions, with a Total Board Power (TBP) of 1,000W. - MI355X: Optimized for liquid-cooled systems, consuming up to 1,400W TBP. The transition to TSMC’s N3P process node for the compute chiplets has significantly increased the transistor count to 185 billion, a 21% jump from the MI300X. This architectural shift, combined with the reduced I/O Die (IOD) from four to two tiles, has enabled AMD to double the Infinity Fabric bus width, improving bi-sectional bandwidth to up to 5.5 TB/s while reducing power consumption in the uncore section. The MI350X series also supports new data types, including FP4 and FP6, alongside the more common FP8 and FP16, which are crucial for AI workloads. Performance Claims AMD claims that its new GPUs can beat Nvidia in certain benchmarks: - Inference: Up to 1.3X faster in like-for-like inference tests against Nvidia's comparable GPUs. - Training: A slight 1.13X performance advantage in select training workloads. For instance, an eight-GPU MI355X setup offers performance ranging from 1.3X faster with four MI355X vs four DGX GB200 in Llama 3.1 405B, to 1.2X faster with eight MI355X against an eight-GPU B200 HGX config in inference in DeepSeek R1. In training workloads, AMD highlights either parity or a slight edge in performance across various LLama models. Rack-Level Solutions AMD’s new MI350X and MI355X GPUs will power rack-level solutions throughout the remainder of the year and into 2026. The company aims to optimize performance-per-TCO (total cost of ownership) by enabling more powerful configurations in a single rack. For liquid-cooled Direct Liquid Cooling (DLC) racks, AMD can fit 128 MI355X GPUs, providing 36TB of HBM3E memory. Air-cooled solutions, however, cap out at 64 GPUs and 18TB of HBM3E memory. Networking and Support AMD is supporting all forms of networking to ensure flexibility and scalability. The new Pollara Ultra Ethernet Consortium-capable NICs (UEC) are positioned as optimal scale-out solutions, while the Ultra Accelerator Link (UAL) interconnect is used for scale-up networking. Despite focusing on AI inference, AMD is working towards unleashing the full potential of rack-scale architectures through strategic acquisitions and partnerships with major OEMs. Industry Insights and Future Projections Mark Papermaster, AMD's Chief Technology Officer, introduced the MI355X accelerators atISC 2025, emphasizing the rapid advancements in supercomputing. He noted that compute performance has historically doubled every 1.2 years, driven initially by CPU-only systems and later by heterogeneous architectures combining CPUs and GPUs. Today's systems, like El Capitan and Frontier, are breaking the 1 ExaFLOP barrier due to specialized AI hardware. However, this performance gain comes with increasing power consumption. The MI300X, released in mid-2023, had a peak power consumption of 750W, while the MI355X will top out at 1,400W. Papermaster projects that future accelerators could consume up to 2,000W by the end of this decade. This trend necessitates substantial improvements in energy efficiency, with AMD aiming to double efficiency every 2.2 years to keep power demands manageable. Without such gains, zettascale systems delivering 1,000× exaflop-class performance could consume 500 MW of power—half the output of a nuclear power plant—making them prohibitively expensive to operate. AMD believes that achieving dramatic performance increases in the next decade will require both architectural breakthroughs and continued advancements in memory bandwidth to complement compute capabilities. Company Profile and Market Position AMD has long been known for its innovative approach to computing architecture, and the launch of the MI350X and MI355X GPUs underscores its commitment to staying competitive in the AI and HPC domains. The company leverages strategic acquisitions and a robust network of partners to enhance its offerings and address the evolving needs of the market. The introduction of these powerful new GPUs marks a significant milestone in AMD’s journey to challenge Nvidia’s dominance and drive the next wave of computational advancements.