High Bandwidth Memory

Overview

High Bandwidth Memory (HBM) is a stacked DRAM architecture that uses through-silicon vias (TSVs) to connect multiple DRAM dies in a vertical stack, placed adjacent to or directly above the compute die on a 2.5D or 3D package. HBM provides bandwidth measured in terabytes per second per device, orders of magnitude higher than conventional DDR DRAM, making it essential for AI accelerator workloads where large model weights and activations must stream through memory at high throughput.

The JEDEC standards body is still finalizing HBM5 as of October 2025. While the standard interface enables multi-vendor interoperability, some hyperscalers and AI chip designers are pursuing Custom HBM — variants with non-standard base dies that embed custom logic or modified electrical interfaces to exceed JEDEC-specified bandwidth limits. Additionally, Compute-in-Memory (CiM) approaches place compute logic inside the HBM stack itself, reducing round-trip memory latency.