Hot Chips 2026: Nvidia presents Groq 3 LPX architecture and unveils its first third-party inference benchmark
Google also provides useful software for agents to get the most out of its hardware. TPU Sync is an open-source “model-state layer for TPU clusters” that moves KV caches off the chip’s HBM, “automatically tiering massive KV caches off the chip in milliseconds” down to host DRAM and, experimentally, shared storage. Google says this can cut KV transfer latency by “more than 90%,” with time-to-first-token gains that vary by workload. The company has also announced the Agent Compute Engine, a Google Cloud service that pauses idle agent sessions and moves their working memory aside until they’re needed again, another memory-saving effort.
Google credits its vLLM-TorchTPU stack with “state-of-the-art price-performance marks” on InferenceX and AgentX, which test realistic agentic coding and long-context work. However, the only published third-party numbers so far are for the last-generation Ironwood TPU. In SemiAnalysis ’s InferenceX tests in September , running Qwen3.5 397B in FP8, Ironwood delivered 50.4% more tokens per dollar than Nvidia’s B200 and 96.0% more than the B300 at 20 tokens per second per user. One issue with that, however, is that Ironwood lacks native FP4, although newer November results show its FP8 beating Nvidia’s FP4 GB300 NVL72 on the same model, at 1.67 times the performance per dollar. There is also no eighth-gen result yet to compare how the new generation differs.
In contrast to the 8i Max, which is designed for long-context work, the new TPU 8tx, “built on the same silicon as TPU 8t,” is “optimized for secure and confidential inference” as well as small-scale training. The security is built into the hardware with isolation to prevent snooping on sensitive data, including model weights. Normally, data has to be unlocked in memory during processing, which is the step this isolation is meant to cover.
This chip is intended for areas where security and confidentiality are high priorities, which means regulated enterprises and governments, particularly those who are using frontier or proprietary models. In that case, the operators may want to “fine-tune models using their private data, distill large models into smaller ones, and train their agents,” without having to move sensitive data between clusters. Agents and security meet here for “secure agent execution.” Back in April, the Pentagon and Google reportedly held discussions for TPU usage in classified settings, although the 8tx chip is not specifically named.
Google’s Private AI Compute, its “hardened TPU platform” with protections “comparable to a typical TEE,” per its September technical brief, already runs confidential serving on TPUs. The 8tx appears to be that capability turned into a product for customers. Alternatives include Nvidia’s NVL72, with one security domain across 72 Rubin GPUs, 36 Vera CPUs, and the interconnects, and Intel’s TDX , which is adding trusted I/O to cover data moving to accelerators.
OpenAI’s Jalapeño ASICs are deployed alongside AMD EPYC ‘Turin’ CPUs as hosts, not Nvidia's Vera
Tesla unveils AI accelerator roadmap, adopts new 'SI' branding in 'Super Intelligence' push
Hot Chips 2026: OpenAI's Jalapeño AI ASIC unpacked
Key considerations
- Investor positioning can change fast
- Volatility remains possible near catalysts
- Macro rates and liquidity can dominate flows
Reference reading
- https://www.tomshardware.com/tech-industry/artificial-intelligence/SPONSORED_LINK_URL
- https://www.tomshardware.com/tech-industry/artificial-intelligence/google-adds-two-new-specialized-chips-for-ai-agents-and-security-to-8th-gen-tpu-lineup-tpu-8i-max-puts-two-chips-on-one-package-for-long-context-work-tpu-8tx-swaps-arm-hosts-for-amd-epyc#main
- https://www.tomshardware.com/my-account
- Cities Worldwide Build and Scale AI Services With NVIDIA Physical and Agentic AI
- Software engineer Rickrolls his Prusa 3D printer via remote code execution exploit
- Software engineer Rickrolls his Prusa 3D printer via remote code execution exploit
- Hobbyist turns $45 USB-C hub into a working Nintendo Switch 2 dock by patching its firmware
- Nvidia’s Board reportedly preparing to oust CEO Jensen Huang as second extra-marital affair emerges
Informational only. No financial advice. Do your own research.