Inside the AI accelerator arms race: AMD, Nvidia, and hyperscalers commit to annual releases through the decade
Graviton4's architecture — 96 Neoverse V2 cores, a CMN-700 mesh, 12 DDR5-5600 channels — operates efficiently with a centralized or semi-centralized SLC. But doubling the core count to 192 dramatically increases mesh traffic, hop distances, and contention on any unified cache structure. At this scale, a monolithic SLC could almost certainly become a latency bottleneck and would not support AWS’s claim of up to 33% lower inter-core communication latency. A distributed L3 sliced across the die allows hot data to remain physically close to compute clusters, reducing average access latency and improving overall coherence behavior.
The fivefold cache expansion AWS advertises reinforces this architectural necessity. Scaling Graviton4's 36 MB SLC by that factor yields 180 MB, and AWS's additional statement —2.6X more cache per core, at double the core count — implies ~187 MB total, which aligns with a large, multi-slice L3 rather than a single SLC block, which would create routing complexity.
Finally, L3-based designs offer stronger multi-tenant performance predictability, which is crucial for AWS. Under cloud workloads, shared caches experience heavy cross-tenant interference and variable latency, so when designing cache subsystems, developers must take into account AWS's use case. To sum things up, the shift to a distributed L3 was a necessary architectural evolution for Graviton5.
Just as AWS didn't disclose many details about other design aspects of Graviton5, it also didn't disclose much about the memory subsystem of the CPU. It goes without saying that Graviton5's memory subsystem is more powerful than that of Graviton4, as it supports higher memory speeds, which likely means that it at least retains a 12-channel memory subsystem of the Graviton4, but with higher data transfer rates (i.e., higher than DDR5-5600).
A 12-channel DDR5 design operating at 6400 MT/s would provide around 614 GB/s of aggregate bandwidth, which translates to approximately 3.2 GB/s per core, which is actually lower than 5.6 GB/s per core in the case of Graviton4. However, the larger L3 cache could compensate for this decrease in memory bandwidth. Then again, we do not know the exact number of memory channels supported by Graviton5.
Input/output throughput is similarly increased, according to AWS: network bandwidth is up by 15% on average across instance sizes, with as much as double the throughput for the largest configurations. Storage bandwidth through Amazon EBS rises by around 20% on average, according to AWS. These gains are designed to improve performance not only for compute-heavy applications, but also for distributed systems that depend on fast storage and networking.
On the security side, Graviton5 is built on the AWS Nitro System, with sixth-generation Nitro Cards that handle virtualization, networking, and storage. AWS has also introduced a new component called the Nitro Isolation Engine, which the company describes as a formally verified isolation layer. Instead of relying solely on conventional security validation, the Isolation Engine uses mathematical proofs to demonstrate that workloads are separated from each other and from AWS operators. The architecture enforces a zero-operator-access model, and AWS plans to allow customers to review the implementation and the formal proofs behind it to ensure maximum security. Such security measures could be a part of the company's effort to attract clients who have traditionally used on-prem servers.
AWS's new Graviton5 processor offers a 192-core, 3nm Arm-based CPU with around 180 MB of L3 cache. This positions the cloud giant as a competitor to the high-end AMD EPYC and Intel Xeon solutions for data centers. The CPU integrates Neoverse V3 cores and delivers an advertised 25% performance uplift, which is conservative given the twofold increase in core count, large microarchitectural improvements in the Armv9.2 ISA, and a fivefold increase in cache capacity. Also, AWS confirms 33% lower inter-core latency due to a redesigned internal layout but has not disclosed whether it uses Arm's CSS, suggesting that Graviton5 may be a unique Annapurna Labs design built around Neoverse V3 cores.
A key architectural shift is replacing Graviton4's SLC with a large distributed L3 to enable better coherence scaling across 192 cores and predictable latency. The processor also gains a faster memory subsystem (likely retaining 12 channels at higher DDR5 speeds), improved network and storage bandwidth, and the new Nitro Isolation Engine, which uses formal verification to guarantee tenant isolation and enforce zero-operator access.
At present, Graviton5 powers new EC2 M9g instances — up to 30% – 35% faster for databases, web services, and machine learning — and compute-optimized C9g and memory-optimized R9g variants will follow in 2026.
Anton Shilov Social Links Navigation Contributing Writer Anton Shilov is a contributing writer at Tom’s Hardware. Over the past couple of decades, he has covered everything from CPUs and GPUs to supercomputers and from modern process technologies and latest fab tools to high-tech industry trends.
Key considerations
- Investor positioning can change fast
- Volatility remains possible near catalysts
- Macro rates and liquidity can dominate flows
Reference reading
- https://www.tomshardware.com/pc-components/cpus/SPONSORED_LINK_URL
- https://www.tomshardware.com/pc-components/cpus/amazon-unveils-192-core-graviton5-cpu-with-massive-180-mb-l3-cache-in-tow-ambitious-server-silicon-challenges-high-end-amd-epyc-and-intel-xeon-in-the-cloud#main
- https://www.tomshardware.com
- New app injects AMD's AI-driven FSR Redstone framegen into unsupported games — pre-alpha OptiScaler build demo shows nice improvements over legacy FSR frame gen
- With a standout $1739.99 price tag, this packed CyberPower PC features both the Ryzen 7 9800X3D and Radeon RX 9070 XT for an amazing prebuilt gaming setup — bea
- Robots’ Holiday Wishes Come True: NVIDIA Jetson Platform Offers High-Performance Edge AI at Festive Prices
- Industry preps new 'cheap' HBM4 memory spec with narrow interface, but it isn't a GDDR killer — JEDEC's new SPHBM4 spec weds HBM4 performance and lower costs to
- Germany summons Russian ambassador over GRU-linked cyberattacks — air traffic control and elections systems targeted
Informational only. No financial advice. Do your own research.