TPU 8i Max doubles TPU 8i compute and HBM bandwidth; TPU 8tx adds confidential inference

Google introduced TPU 8t for training and TPU 8i for reasoning at Cloud Next in April. With these two additions it has introduced four TPU systems in one year. Amin Vahdat's post describes the new parts as systems rather than new chips: existing silicon with different packaging, memory, networking and security around it. For teams on Google Cloud, the practical question is which of the four fits a given workload. The announcement answers that only in relative terms.

Google aims the dual-chip system at the memory wall. Its example is dozens of AI agents working across an entire codebase or hours of video at once, where memory bandwidth quickly becomes the bottleneck. Its answer is to scale compute and memory in lockstep rather than add one without the other.

The stated result is 2x compute, 2x SRAM and 2x HBM bandwidth relative to TPU 8i, with twice the silicon in the existing rack footprint. Google says that gives significantly more computing power per square foot and per watt, and calls TPU 8i Max the highest compute density per rack in the lineup. The post does not say whether HBM capacity also doubles. It also gives no absolute capacity or bandwidth figure for TPU 8i or TPU 8i Max.

For KV-cache-heavy, million-token serving, capacity is the number that decides how many concurrent long contexts fit on a device, and it is missing. Google's software answer to capacity is TPU Sync. It treats chip HBM, host DRAM, shared storage and the data center network as one memory hierarchy and tiers KV caches off the chip. More on that below.

TPU 8tx uses the same silicon as TPU 8t, paired with 5th Generation AMD EPYC CPU hosts. Google says it keeps sensitive data and model weights protected and isolated even while in active use, not only when stored or sent over a network. It adopts the low-latency Boardfly network Google created for real-time reasoning. Google claims up to 2x lower latency than TPU 8t with a near-zero performance penalty. The post does not give the workload behind either figure.

The engineering case is consolidation. Google positions TPU 8tx for regulated enterprises and governments, covering confidential inference and small-scale training on one system: private fine-tuning, distillation into smaller models, and reinforcement learning on agents. Sensitive data does not have to move between separate clusters, which removes one data-movement boundary from a compliance review. The post does not describe the attestation model, key management, or which parts of the stack sit inside the trust boundary.

Every generational number in the post is relative. Google says TPU 8t delivers up to 2.7x better training performance and TPU 8i 80% better inference price-performance than its previous generation of TPUs. It also says TPU 8i cuts inter-chip communication delays by up to half. The post never names TPU 7 or any previous-generation part, and it gives no peak FLOPS, precision formats or memory figures for any eighth-generation system. Because TPU 8i Max and TPU 8tx are defined against TPU 8i and TPU 8t, a team can only place them by chaining ratios back to hardware it may not have benchmarked.

One Hacker News commenter asked for watt-per-token and dollar-per-token figures for each system and how each compares with its Nvidia equivalent. The post has neither, and it contains no Blackwell or other GPU comparison. Its closest substitute is a claim of state-of-the-art price-performance on the InferenceX and AgentX benchmarks for the vLLM-TorchTPU stack, without the scores or the TPU generation used. Other commenters described the piece as a marketing announcement with no specs. On the hardware, that is accurate.

Google gives no chip counts per pod or slice for the new systems. What it does describe is how slices are formed. GKE dynamic slicing is now generally available. It uses optical circuit switching to combine TPU cubes into larger topologies (super-slicing) or split them into isolated smaller ones (sub-slicing) without recreating nodepools. Google claims job startup up to 5x faster and fault recovery up to 4.5x faster.

Cloud Pathways, also now generally available on GKE, provides a single-controller runtime. It orchestrates multiple slices as one machine for pre-training, RL and multihost inference, with colocated Python sidecars, sub-minute slice hot-swapping and elastic resharding. For anyone sizing a deployment, these describe how flexibly capacity can be carved. They do not say how large a single slice can get.

For GPU-first teams, the software list matters more than the silicon:

The top Hacker News complaint was that a section titled 'Opening up TPUs to every developer' offered no information on how an individual developer can start. Another commenter noted that the largest TPU on Google Colab is still v6e-1. The post links to TorchTPU and TPU Sync but names no instance types, quotas or regions for TPU 8i Max or TPU 8tx.

The post gives no general-availability date, preview window, regions or pricing for either new system. Instead it points readers to a broadcast and the Google Cloud website. It does say Google's TPU business has nearly 100 customers, including frontier labs, trading firms, retailers and universities, and that customers can now deploy TPUs in their own data centers. It does not say whether TPU 8tx or TPU 8i Max will be offered on-premises. Until Google publishes memory capacity, slice sizes and per-hour pricing, treat both as roadmap items rather than capacity you can reserve.