HyperAIHyperAI

Command Palette

Search for a command to run...

Google
LLM

Google Launches TPU 8tx and 8i Max for Agentic Era

Google Cloud has expanded its eighth-generation TPU portfolio with two new systems, TPU 8i Max and TPU 8tx, marking the first time the company has introduced four TPU systems in a single year. The additions build on the TPU 8t and TPU 8i platforms announced earlier in the year, which delivered up to 2.7 times better training performance and 80 percent better inference price performance than the prior generation. The company said the eighth generation now covers distinct stages of the agentic AI lifecycle: TPU 8t for scale-up training, TPU 8i for low-latency reasoning, TPU 8tx for confidential computing and TPU 8i Max for massive context windows and the highest compute density per rack. TPU 8tx is designed for secure and confidential inference and small-scale training, targeting regulated enterprises and government customers running frontier or proprietary models with sensitive data. Built on the same silicon as TPU 8t and paired with fifth-generation AMD EPYC hosts, the system embeds security at the hardware layer, keeping sensitive data and model weights protected during active processing. Google said it uses the low-latency Boardfly network developed for real-time reasoning, delivering up to two times lower latency than TPU 8t with near-zero performance penalty. The design allows organizations to fine-tune models, distill large models and train agents without migrating sensitive data between separate clusters. TPU 8i Max focuses on the growing need for accelerators that can reason across very large model contexts, including million-token workloads, while maximizing compute density per rack. Google said the broader push reflects the shift from human-scale to agent-scale computing, with external customers increasingly asking for specialized hardware capabilities and software accessibility. The company said TPUs now serve nearly 100 customers, including frontier labs, trading firms, retailers and universities. The hardware announcements were paired with a broader effort to make TPUs easier to use across the developer ecosystem. Google highlighted the open-source TorchTPU project, which enables PyTorch code and Hugging Face models to run natively on TPUs, along with support for serving engines such as vLLM and SGLang. A new TPU agent suite aims to automate onboarding, diagnose bottlenecks and tune kernels through plain-language interaction with coding tools. Google also introduced an open-source XLA Serving Engine for low-latency production serving across TPUs, GPUs and CPUs. On the infrastructure side, Google is targeting the operational inefficiencies created by stateful AI agents. The new Agent Compute Engine software suspends idle agent sessions and moves working memory off the chip in milliseconds to reduce power consumption and infrastructure cost. TPU Sync, an open-source model-state layer, unifies high-bandwidth memory, host memory, storage and networking into a single tiered memory system to accelerate million-token context handling and reinforcement learning workflows, reducing key-value cache transfer latency and time to first token by more than 90 percent. Google also made Cloud Pathways generally available on Google Kubernetes Engine, allowing customers to orchestrate large TPU systems across multiple slices as if they were a single machine. GKE dynamic slicing is also now generally available, using optical circuit switching to form custom TPU topologies on demand and reduce job startup times by up to five times and fault recovery by up to 4.5 times. The company emphasized that shipping four TPU systems and an open software stack in one year reflects close co-design across silicon, networking, compilers, Google DeepMind research and AI-assisted chip development.

Related Links