Technical Paths to the New Era of GPU-Initiated Storage

At the 2025 OCP Global Summit, CJ Newburn and Vikram Mailthody present technical paths to a new era of GPU-initiated storage: reinventing the GPU so it directly initiates fine-grained, massively concurrent access to NVMe storage independent of the CPU. They argue that emerging data-intensive workloads (vector databases, GNNs, recommendation, predictive AI) have working sets too large for memory and demand IOPS-per-dollar-optimized SSDs and storage disaggregation. The talk introduces the SCADA (Scaled Accelerated Data Access) interface and shares benchmarks where GPUs reach far higher IOPS than CPUs, closing with the OCP Storage Next standardization effort.

Key Takeaways

Emerging data-intensive workloads (vector search, GNN traversal, recommendation, predictive AI) have working sets too big for memory and require accessing storage like memory but through an API, with about 1000x more concurrency than a CPU can provide.
GPUs are the right engine for this because they offer roughly 100,000 threads versus about 100 on a CPU, have HBM, and tolerate latency well; the whole request path (cache use, request preparation across NVMe, request execution) runs independent of CPUs because CPUs cannot keep up on concurrency.
SCADA (Scaled Accelerated Data Access) is the proposed GPU-initiated interface: it scales to 100K threads, supports errors/retries since memory fabric breaks at scale, allows taking drives offline and back, and lets unchanged apps (cuGraph WholeGraph, cuVS) gain a GPU-side specialized cache.
Storage disaggregation is needed because a single GPU can demand ~200 million IOPS at half-KB accesses (30+ discs), which cannot fit in a compute node; disaggregated pools allow specialized, shareable storage nodes.
Benchmarks: an H100 (Gen5) reached 96 million IOPS at only ~24% GPU utilization on an unoptimized stack versus 43-45 million IOPS for a same-generation CPU with fio/io_uring/SPDK; an L40 (Gen4) hit 48 million IOPS (the Gen4 peak); with Micron 9650 8TB Gen6 drives, scaling to 48 drives reached close to 250 million IOPS.
On an end-to-end GNN training workload (WholeGraph, MLPerf-style dataset), keeping all node-embedding data only on NVMe was just 20% slower than the all-CPU-memory case, surprising even the presenters.
Legacy SSDs optimize capacity per dollar, but fine-grained sparse access needs a new IOPS-per-dollar/TCO-optimized SSD class; the OCP Storage Next effort gathers stakeholders to define and standardize these drives.