The custom Instinct MI455X for Meta will carry 144GB of HBM4 memory using six 8-Hi packages, whereas the full-blown Instinct MI455X will be equipped with 432 GB of HBM4 memory, according to SemiAnalysis . In addition, the part will reportedly offer 'significant decreases in compute.' The new design will offer a more competitive bandwidth-per-dollar ratio for recommendation systems, but will not be optimized for training of frontier AI models or running inference, the report claims.
Cutting compute performance and reducing HBM4 capacity from 432GB to 144GB should dramatically reduce the bill of materials, as HBM4 is exceptionally expensive. Furthermore, the reduction would cut the package size of the custom Instinct MI450-series accelerator for Meta, which is another way to reduce BOM costs. By using custom cut-down Instinct MI450-series accelerators instead of fully-fledged models, Meta can potentially save tens of millions of dollars.
As added bonuses, these custom Instinct MI450-series accelerators will also consume significantly less power when running recommendation workloads without significantly reducing performance. Also, such accelerators can offer better CPU/GPU balance for recommendation systems, according to SemiAnalysis . If Meta runs these accelerators primarily on recommendation workloads for their entire useful lives, the custom design could deliver substantially better total-cost-of-ownership.
However, such cutting down has many disadvantages. The biggest problem is loss of versatility. The reductions in both compute and HBM make it less attractive for LLM training and inference. The standard Instinct MI455X has 432 GB of HBM4 and 19.6 TB/s of bandwidth, which is particularly beneficial for large-scale training and inference. By contrast, the 144 GB capacity may be particularly restrictive for modern LLM training and inference.
AMD announces MI350P PCIe AI accelerator card with 144GB of HBM3E — roughly 40% faster in FP16 and FP8 theoretical compute compared to Nvidia's H200 NVL competitor
AMD to supply Anthropic with 2 gigawatts of Instinct MI450 GPUs
Key considerations
- Investor positioning can change fast
- Volatility remains possible near catalysts
- Macro rates and liquidity can dominate flows
Reference reading
- https://www.tomshardware.com/tech-industry/artificial-intelligence/SPONSORED_LINK_URL
- https://www.tomshardware.com/tech-industry/artificial-intelligence/meta-to-use-custom-amd-instinct-mi400-accelerators-with-144gb-of-hbm4-for-select-workloads-report-claims-could-dramatically-reduce-cost-at-the-expense-of-versatility#main
- https://www.tomshardware.com/subscription
- PlayStation 3 emulator adds support for ATI Radeon HD 2000, 3000, and 4000 series graphics cards on Linux — 20-year-old HD 2600 crumbles, can only run Portal at
- Built in Fort Worth: Wistron Opens Advanced Manufacturing Plant to Produce NVIDIA AI Systems
- NVIDIA AI Supercomputer Comes Online at Naval Postgraduate School
- Inside optical and the battle for scale – how the AI industry is racing to integrate photonic interconnects
- AMD takes the wraps off its Instinct MI455X AI accelerator — CDNA 5 and Helios rack-scale architecture combine to take the fight to Nvidia in the data center
Informational only. No financial advice. Do your own research.