Tech titans team up to form optical interconnect alliance to solve the AI buildout’s big data bottleneck — Nvidia, AMD, Broadcom & more set sights on building P

by

Tech titans team up to form optical interconnect alliance to solve the AI buildout's big data bottleneck — Nvidia, AMD, Broadcom & more set sights on building P

When you purchase through links on our site, we may earn an affiliate commission. Here’s how it works .

This week, AMD, Broadcom, Nvidia, OpenAI, Meta, and Microsoft announced plans to standardize a protocol-agnostic, scale-up interconnect for AI data centers. The Optical Compute Interconnect Multi-Source Agreement (OCI MSA) group is tasked with defining an open connectivity specification for optical interconnections in AI data centers. This would allow for higher domain scale-up sizes, and enable a multi-vendor supply chain for optical interconnects, which the ongoing AI infrastructure buildout demands.

The group's primary goal is to enable data centers to scale by using optical interconnections rather than relying solely on copper, which is currently hitting its physical limits for optimal data transfer speeds and power consumption. Copper is also facing significant supply chain constraints , and an industry-wide shift to optical interconnections would alleviate some of this demand. An optical interconnection would also bolster data transfer speeds, crucial for large-scale AI workloads.

For copper, pushing electrical signals to high speeds results in signal degradation and unsustainable levels of power consumption; the solution is optical interconnects. Copper is inherently a lossy, resistant medium, necessitating huge amounts of power to send data over distances at high speeds. An optical physical layer (PHY) can overcome this electrical resistance challenge, allowing for higher-speed data transmission. The goal of the newly-founded group is to develop a PHY capable of delivering up to 3.2Tb/s and beyond.

You may like AMD, Broadcom, and Nvidia join hyperscalers to define optical scale-up interconnect of the future for AI clusters Photonics and high-speed data movement is the next big AI bottleneck Start-up plans to use terahertz radio frequencies for communication between servers instead of copper or optical connections Given that power is an ongoing challenge in AI data center buildouts, a solution that not only stabilizes power usage but also increases interconnection speeds seems like an obvious choice.

Optical cables would let more systems be concurrently connected over greater distances, without many of the penalties that come with copper, improving scale-up domains. However, optics also comes with its own downsides: Failure rates, increased heat output, higher overall costs, and overall failure rates. With the technology nascent in its application, new standards must be created in order for it to mature.

"Optical cables and the silicon photonics technology already exist when it comes to connecting different switches as part of a pluggable transfer ecosystem.” said Vivek Raghunathan, CEO and co-founder at Xscape Photonics, in a recent interview with Tom's Hardware Premium . Indeed, TSMC's COUPE technology is a foundational bedrock for enabling optical and photonics in chips; where the new OCI MSA standard comes in is to enable these physical chips to effectively travel across the same lanes.

The open standard developed by the OCI MSA would allow multiple vendors within the optical supply chain to offer components to a singlular, unified spec. In theory, it should drive down the cost of optics and optical interconnects at scale. Moreover, it divests the sole reliance on TSMC, as the standard would ensure interoperability between products and chips made with COUPE, and those using alternative CPO packaging platforms, effectively de-risking optical supply chains in the process.

If co-packaged optics were introduced into data centers without an effective rulebook or guidelines to govern them, each vendor design would become proprietary, siloed between one another. So for larger-scale data centers that run multiple ecosystems, the OCI MSA's new spec defines a standard to let them drive on the same roads and use the same data pathways. This isn't about hooking together systems and getting them to understand each other via NVLink or UALink , but instead about offering a physical foundation to allow those protocols to travel over fibre connections.

Within a data center, the OCI MSA's newly defined standard would allow a data center to theoretically run NVLink for Nvidia chips and UALink for AMD hardware, while making use of the same underlying optical infrastructure, at higher speeds than copper can realistically allow.

“Fundamentally, the current copper-based interconnects just cannot meet that bandwidth requirement," Raghunathan continued. The OCI MSA's standard supports a range of optical solutions, such as pluggable optical modules, on-board optics, and chips using co-packaged optics. This would effectively break the barriers that copper currently employs.

Nvidia invests $4 billion into photonics firms in a bid to bolster data center interconnect supply chains

UALink roadmap plots course to optimized AI data center interconnects

Key considerations

  • Investor positioning can change fast
  • Volatility remains possible near catalysts
  • Macro rates and liquidity can dominate flows

Reference reading

More on this site

Informational only. No financial advice. Do your own research.

Related posts:

  1. Photonics and high-speed data movement is the next big AI bottleneck — following copper, power, DRAM, and NAND
  2. Photonics and high-speed data movement is the next big AI bottleneck — following copper, power, DRAM, and NAND
  3. Photonics and high-speed data movement is the next big AI bottleneck — following copper, power, DRAM, and NAND
  4. Why copper markets are feeling the pinch due to aggressive AI data center expansion — expansive buildout to demand 1.1 million tonnes of copper annually by 2030

Leave a Comment