NVIDIA Expands NVLink Fusion with NVHBM to Accelerate Next-Gen AI Infrastructure
- NVIDIA expanded its NVLink Fusion platform with a new custom high-bandwidth memory technology called NVHBM, designed to address performance bottlenecks in next-generation artificial intelligence infrastructure.
- Traditional high-bandwidth memory architectures place the memory controller directly on the accelerator or XPU die, which consumes valuable silicon area that could otherwise support additional computing logic.
- Amazon’s Annapurna Labs is the first partner to work with NVIDIA on the NVHBM technology as part of a broader collaboration surrounding the NVLink Fusion architecture.
NVIDIA expanded its NVLink Fusion platform with a new custom high-bandwidth memory technology called NVHBM, designed to address performance bottlenecks in next-generation artificial intelligence infrastructure. According to company announcements, the memory technology integrates a custom memory controller directly into the 3D high-bandwidth memory stack rather than the accelerator die, delivering up to 30% greater memory bandwidth and 15% lower power consumption compared with standard HBM4e configurations.
Architectural Design and Performance Metrics of NVHBM
Traditional high-bandwidth memory architectures place the memory controller directly on the accelerator or XPU die, which consumes valuable silicon area that could otherwise support additional computing logic. By shifting the memory controller into the HBM base die, NVHBM frees up to 25% more area on the XPU compute die, according to NVIDIA’s technical specifications. Training, inference, and agentic AI workloads increasingly depend on high-throughput access to model weights, key-value caches, and activation data. NVHBM helps resolve the memory bandwidth bottleneck by delivering up to 30% more memory bandwidth per stack than standard HBM4e. These improvements allow custom accelerators to read key-value cache data faster, improve training efficiency, and increase per-user token throughput during large-model inference.
Amazon Web Services and Annapurna Labs Collaboration
Amazon’s Annapurna Labs is the first partner to work with NVIDIA on the NVHBM technology as part of a broader collaboration surrounding the NVLink Fusion architecture. According to the companies, Annapurna Labs will support NVLink Fusion with its next-generation Trainium chips, starting with Trainium4, allowing Amazon chips and NVIDIA GPUs to work together within a common rack-scale architecture.
NVHBM represents a new architectural approach to advancing high-bandwidth memory performance and efficiency,
said Nafea Bshara, vice president of Annapurna Labs at Amazon, in a statement provided by NVIDIA. We look forward to this technology collaboration to benefit future AWS infrastructure designs.
Ecosystem Integration Through NVLink Fusion
NVHBM is offered as part of the broader NVIDIA NVLink Fusion platform, which allows hyperscalers and artificial intelligence innovators to connect custom XPUs and central processing units to rack-scale platforms. By establishing a standard NVHBM implementation available from multiple memory providers, NVIDIA aims to reduce the engineering effort required to qualify and integrate memory across diverse suppliers. NVLink Fusion customers gain access to NVLink chiplets, NVLink-C2C, NVLink Switches, and MGX systems and racks. This ecosystem access allows custom silicon designers to focus engineering resources on XPU innovation while utilizing a proven technology stack for scale-up and scale-out networking.
