Storage Disaggregation

Overview

Storage disaggregation moves storage out of the compute node into specialized, shareable pools to meet GPU IOPS demands that cannot physically fit locally. Keeping up with the IOPS of even a single GPU can require around 200 million IOPS per GPU at half-kilobyte accesses, which could take on the order of 30-plus discs today — so the storage cannot fit inside a compute node holding eight GPUs (you cannot put 32 drives per GPU in such a node). Disaggregation lets storage nodes be specialized with extra forms of acceleration, mixes capacity-per-dollar nodes (e.g. for KV cache) with high-IOPS nodes, and creates shareable drive pools that can be allocated to subsets of cluster nodes without onerous up-front data sharding.