Germany Unveils Exascale "Jupiter" Supercomputer: A Hybrid CPU-GPU Marvel with Efficient Performance and Energy Use

Germany's Forschungszentrum Jülich (FZJ) recently unveiled its exascale supercomputer, "Jupiter," which is set to make a significant impact in the high-performance computing (HPC) community. The system is the first exascale supercomputer to be completed under the EuroHPC Joint Undertaking of the European Union, a project aimed at advancing Europe’s independence in HPC technology. However, the current configuration of Jupiter reflects the ongoing challenges faced by the region in achieving chip independence, as it heavily relies on hardware from non-European companies, primarily Nvidia. Configuration and Modules Jupiter is designed as a hybrid system, combining CPU and GPU compute capabilities with advanced storage solutions. The modular design includes a Universal Cluster and a GPU Booster module, both built on the BullSequana XH3000 system design by Eviden, the HPC division of Atos. ParTec, a German HPC system designer and installer, acts as the prime contractor, bringing additional installation and services to the project. Universal Cluster Nodes: Over 1,300 nodes, each equipped with a pair of SiPearl’s Rhea1 Arm CPUs, featuring 80 cores based on the Neoverse V1 architecture. Memory: 64 GB of HBM memory per CPU, providing fast access speeds similar to those found in GPUs. Performance: Expected to deliver approximately 7 petaflops of peak theoretical FP64 performance, contributing a relatively small portion to the overall exascale goal. Purpose: Primarily for traditional CPU-intensive workloads, offering a step towards European HPC independence through the use of SiPearl CPUs. GPU Booster Module Nodes: The heart of Jupiter, consisting of a unique four-way clustering of Nvidia Grace G100 Arm CPUs, each linked with four Hopper H200 GPUs. Compute Nodes: Each node contains two sleds, each with a quad of Grace-Hopper modules, allowing for efficient communication and memory sharing among the CPUs and GPUs. Memory and Bandwidth: Each H200 GPU has 96 GB of HBM3 memory with 4 TB/s of bandwidth. The NVLink 4 ports provide 300 GB/s of inter-GPU bandwidth and 600 GB/s of CPU-GPU bandwidth. Interconnect: Dual-port 400 Gb/s ConnectX-7 NDR InfiniBand cards ensure robust connectivity within and between nodes. Total Compute: The booster module currently houses around 24,000 Grace-Hopper superchips, though the exact count is still being confirmed. This setup is expected to deliver close to 1 exaflops of FP64 performance on the High Performance LINPACK (HPL) benchmark. Performance and Efficiency Initial tests placed Jupiter’s GPU Booster module at number four on the June 2025 Top500 rankings, highlighting its computational prowess. The module achieved 930 petaflops of peak performance (Rpeak) and 793.4 petaflops of HPL performance (Rmax). However, achieving the full exascale target of 1 exaflops on HPL would require the addition of approximately 6,277 more Grace-Hopper nodes, based on current calculations. The computational efficiency of the Jupiter booster module is notably high at 85.3 percent. This is significantly better than other recently installed HPE clusters, such as those at the University of Bristol (77.7 percent) and Sigma2 (78.6 percent), which used the Slingshot Ethernet interconnect. The efficiency and effectiveness of the Nvidia InfiniBand network in Jupiter’s design are evident. Energy Efficiency Jupiter also stands out for its energy efficiency. The JEDI testbed, a precursor to the full Jupiter system, holds the top spot on the Green500 supercomputer energy efficiency rankings, achieving 72.7 gigaflops per watt. Despite the increased complexity of networking a larger number of GPUs, the Jupiter GPU Booster module maintains a respectable 60 gigaflops per watt, which is comparable to the energy efficiency of other leading exascale systems like Frontier (62.7 gigaflops per watt) and El Capitan (58.9 gigaflops per watt). Storage Solutions Flash Storage: A 29 PB raw capacity flash array with 21 PB of usable scratch storage, delivering 2 TB/s write and 3 TB/s read performance. Disk Storage: A separate 300 PB Storage Module, likely based on disk drives, providing ample space for large datasets. Tape Library: An additional 700 PB of tape storage for archival purposes. Note: The disk and tape storage systems were procured separately but are integrated with Jupiter. Modular Datacenter Design The Jupiter system is housed in a highly efficient modular datacenter with advanced cooling solutions. The facility features a honeycomb architecture that allows for optimal air flow and heat dissipation, crucial for maintaining operational reliability and energy efficiency. Funding and Cost Analysis The core funding for Jupiter totals €500 million (approximately $576.1 million at current exchange rates), broken down as follows: - EuroHPC Contribution: €250 million - German Federal Ministry of Education and Research: €125 million - State of North Rhine-Westphalia: €125 million Eviden and ParTec received €273 million ($314.7 million) for hardware, software, and services, while the remaining €227 million ($261.4 million) covered power, cooling, and operations. Given the list price of $22,500 per H200 GPU, the cost of the GPUs alone could reach $670.8 million. This suggests that Nvidia provided a substantial discount to FZJ, similar to the deals made by AMD for Frontier and El Capitan. Such discounts are often necessary to seed the market and encourage adoption of new technologies. Industry Evaluation and Company Profiles Industry insiders commend Jupiter’s innovative architecture, particularly the quad clustering of Grace-Hopper modules, which enhances computational efficiency and energy performance. The use of European-designed SiPearl CPUs in the Universal Cluster is seen as a crucial step towards European HPC independence, despite the dominance of non-European components in the overall system. Eviden: As the HPC division of Atos, Eviden initially planned to spin out but decided to stay within the parent company, emphasizing its commitment to HPC projects. Eviden provided the hardware and system integration for Jupiter, leveraging its experience with the BullSequana XH3000 platform. ParTec: A German HPC system designer, ParTec brings local expertise and services to the table, ensuring that a significant portion of the work and value remains within Germany. Their role as the prime contractor underscores the project’s focus on fostering European HPC capabilities. SiPearl: A French semiconductor company, SiPearl is developing Arm-based CPUs specifically for HPC applications. The inclusion of Rhea1 chips in Jupiter is a milestone for the company, demonstrating its products’ readiness for exascale workloads and contributing to Europe’s push for technological sovereignty. In summary, Jupiter represents a major leap forward in European HPC, blending cutting-edge Nvidia technology with a strategic nod to local innovation. While the current configuration may not fully meet the exascale target, the system’s performance and energy efficiency are impressive, and the modular design allows for future expansions and optimizations.