Researchers turn HBM on its side to tackle AI memory’s heat wall — Korean V-Die and Japanese MOSAIC designs promise higher bandwidth, denser stacks, and cooler

by

Researchers turn HBM on its side to tackle AI memory’s heat wall — Korean V-Die and Japanese MOSAIC designs promise higher bandwidth, denser stacks, and cooler

SoftBank subsidiary working with Intel to develop radical new ZAM memory is now receiving Japanese gov't subsidies

Cooling is the other half of the V-Die argument. The proposal places microfluidic cooling channels between adjacent upright DRAM dies, allowing coolant to dissipate heat closer to its source. According to the researchers, this could keep the stack around 45°C, far below the 80°C-plus range associated with dense HBM systems. In a simulated 16-die stack matched to H100-class hardware on a GPT-3-scale model, V-Die hit 540 tokens per second, compared to HBM4's 296, and cut first-token latency by 32%, or about 24 milliseconds.

MOSAIC, meanwhile, is focused on making the sideways stack manufacturable. Because the dies are assembled flat and then turned on edge, even a few microns of die-thickness variation across dozens of dies can add up to an alignment miss where the signal pads no longer land. The Japanese team’s answer is a contactless interface based on inductive coupling. One side of the memory die carries oblong coils, while a corresponding set of coils sits on the substrate or mating chip. Current in one coil induces a signal in the other, allowing data to cross the small gap without a direct metal-to-metal signal contact. This eliminates the need for precise overlapping, giving the package greater tolerance for assembly variation. Power, which requires fewer, larger connections than data, can still be supplied via physical contacts on the sides of the memory cube.

The VLSI MOSAIC prototype achieved up to 4 Gbps per channel and demonstrated TSV-free 3D integration for a memory-on-GPU layout. The team says the approach can enable twice the memory capacity of HBM4 without significantly increasing peak temperature. A related bump-MOSAIC hardware demonstration at ECTC used 100-micron-pitch microbumps, achieved stacking alignment within 6 microns as verified by X-ray CT, and showed a configuration with three times the thermal conductivity of conventional stacking while adding up to 30% more memory capacity.

While the results look promising, neither V-Die nor MOSAIC is close to replacing commercial HBM . Neither is close to shipping. V-Die is still a proposed architecture, with a prototype in the works to validate its thermal and electrical behavior; MOSAIC has proof-of-principle hardware, but the researchers have yet to show it scales to commercial DRAM capacity, yield, cost, and reliability.

Still, any viable solution to the multifaceted AI memory problem is a welcome development. SoftBank and Intel’s Z-Angle Memory (ZAM) and NEO Semiconductor’s 3D X-DRAM — both still in development — aim to solve the constraints of conventional memory. Meanwhile, the overall market is already feeling the squeeze on price and availability, even as memory makers divert capacity toward the more lucrative AI HBM and server products, driving consumer RAM prices even higher.

Follow Tom's Hardware on Google News , or add us as a preferred source , to get our latest news, analysis, & reviews in your feeds.

Etiido Uko is a news contributor for Tom's Hardware covering the latest updates in big tech and the PC industry. He is a mechanical engineer and senior technical writer with over nine years of experience in documentation and reporting. He is deeply passionate about all things engineering and technology, and is an expert in gadgets, manufacturing, robotics, automotive, and aerospace. ","collapsible":{"enabled":true,"maxHeight":250,"readMoreText":"Read more","readLessText":"Read less"}}), "https://slice.vanilla.futurecdn.net/13-4-25/js/authorBio.js"); } else { console.error('%c FTE ','background: #9306F9; color: #ffffff','no lazy slice hydration function available'); } Etiido Uko Social Links Navigation News Contributor Etiido Uko is a news contributor for Tom's Hardware covering the latest updates in big tech and the PC industry. He is a mechanical engineer and senior technical writer with over nine years of experience in documentation and reporting. He is deeply passionate about all things engineering and technology, and is an expert in gadgets, manufacturing, robotics, automotive, and aerospace.

Key considerations

  • Investor positioning can change fast
  • Volatility remains possible near catalysts
  • Macro rates and liquidity can dominate flows

Reference reading

More on this site

Informational only. No financial advice. Do your own research.

Related posts:

  1. Researchers turn HBM on its side to tackle AI memory’s heat wall — Korean V-Die and Japanese MOSAIC designs promise higher bandwidth, denser stacks, and cooler
  2. Chinese semiconductor industry gears up for domestic HBM3 production by the end of 2026 — CXMT to produce chips, while Naura, Maxwell, and U-Preseason design to
  3. Chinese semiconductor industry gears up for domestic HBM3 production by the end of 2026 — CXMT to produce chips, while Naura, Maxwell, and U-Preseason design to
  4. Chinese semiconductor industry gears up for domestic HBM3 production by the end of 2026 — CXMT to produce chips, while Naura, Maxwell, and U-Preseason design to

Leave a Comment