Token Bucket Rate Limiting

Overview

Token bucket rate limiting is an algorithm for implementing SSD QoS rate limits. A bucket starts with tokens, adds tokens at a constant rate up to a maximum (excess tokens overflow and are discarded), and each arriving command must consume tokens to proceed. It is presented as one possible, non-mandatory method to implement NVMe TP4176 rather than a required part of the specification.

A defining property distinguishes SSD rate limiting from networking: unlike a network, an SSD cannot drop commands when tokens are exhausted. It must queue every command and return each one — at minimum with an error — never losing track of them, which makes SSD rate limiting a form of traffic policing rather than traffic shaping. The token bucket applies independently to IOPS and bandwidth: for IOPS, a write decrements one token while the total is decremented by the Write-to-Read IOPS Scalar; for bandwidth, the bucket is decremented by the write size while the total is decremented by the write size times the Write-to-Read BW Scalar.