Deep-Buffer Networking

Overview

Deep-buffer networking refers to switch and router architectures that provide large per-chip packet buffer capacity — orders of magnitude more than shallow-buffer commodity Ethernet switches — to absorb traffic bursts without dropping packets. The architecture is common in traditional enterprise routers and wide-area networking equipment.

For AI data center networking, deep buffers are controversial. Proponents argue they are necessary; critics claim they introduce jitter that degrades AI training job completion times. Cisco addressed both objections at OCP Global Summit 2025:

Deep buffers do not slow AI workloads: AI training is fully synchronous — all workers in a collective communication must complete before the next iteration begins. Job completion time is therefore set by the slowest path, not the average jitter. Deep buffers at a well-managed switch with proper congestion control and load balancing are not the root cause of slow paths; congestion itself is.
Deep buffers are not inefficient: Buffers only fill during congestion events. The presence of buffer capacity does not consume power proportional to its size when idle. The inefficiency argument conflates buffer provision with buffer utilization.

For scale-across networks specifically, deep buffers are necessary because: (a) at WAN-scale distances, link failure is the norm rather than the exception at cluster scale; (b) roundtrip times grow with fiber distance (the "tyranny of distance"), meaning reactive congestion control signals arrive too late to prevent queue buildup during failure rerouting; (c) deep buffers absorb the reactive traffic surge during the rerouting window, preventing packet loss.