GPU-Initiated Storage

Overview

GPU-initiated storage is a model in which the GPU directly initiates fine-grained, massively concurrent access to NVMe storage, independent of the CPU. It is motivated by emerging data-intensive workloads — vector databases, graph neural networks, recommendation, predictive AI — whose working sets are too big to fit in memory, requiring storage to be "accessed like memory, but with an API," at roughly 1000x the concurrency a CPU's thread count can provide. The GPU is the right engine because it offers about 100,000 threads versus roughly 100 on a CPU, has HBM, and is very tolerant of latency. The access logic is placed entirely on the GPU and splits into three categories of client requests: use a cache when possible; batch large numbers of requests (16k-32k) into a buffer that many threads contribute to; then prepare the request by distributing it across NVMe drives and execute it with write buffering and data-integrity management. The entire request path runs independently of CPUs because CPUs cannot keep up — the fundamental limitation being concurrency. Data-intensive workloads can require on the order of 10 million-plus IO transactions between two synchronization points, a level of concurrency not achievable on typical CPUs.