Scale-Out Networking

Overview

Scale-out networking is the backend AI fabric that connects multiple GPU racks within a single data center. AI backend networks sustain approximately 95% network utilization — compared to under 50% for general-purpose networks — creating extreme demands for intelligent load balancing and congestion management. Communication stall during distributed training averages 30% of total training time, and for some model architectures network communication consumes over 50% of combined compute and communication time.

Standard Ethernet with RoCEv2 suffers from three structural problems at AI scale: ECMP hash collisions that underutilize links by mapping many flows to the same paths (tor-to-spine); PFC-based lossless transport that creates deadlock/livelock problems at scale; and go-back-N retransmission that degrades performance whenever any packet loss occurs. Ultra Ethernet addresses all three with intelligent load balancing, lossy congestion management, selective acknowledgement, and fast failover.