Insights from Production: Scheduled Ethernet Fabric in Large AI Training Clusters

Author: Pengfei Huo, Sr. Network Architect, ByteDance; Henry-Xiguang Wu, Switch Product Marketing, Broadcom Collection: 2024 OCP Global Summit - Networking

Summary

This presentation details ByteDance's journey from evaluating scheduled Ethernet fabric technology in 2022 to deploying it in production in June 2024, co-presented with Broadcom who provides the underlying Jericho and Ramon silicon. Scheduled Ethernet fabric is an OCP-compliant system using receiver-based scheduling where all traffic must get explicit permission from the receiver before traversing the fabric, eliminating congestion and collisions. Leaf switches use the Broadcom Jericho family with standard Ethernet interfaces to GPUs, while spine switches use the Ramon family for highly power-efficient forwarding.

ByteDance's first production deployment supports 1,280 xPU cards at 400GE speeds in a two-layer architecture with 20 NCPs (leaf) and 20 NCFs (spine), running a distributed management system without any centralized controller (NCC). Performance results from the production cluster using 128 GPU cards show scheduled fabric outperforming typical five-tuple hash Ethernet by 22% on Reduce Scatter, 21% on All Gather, and 119% on All Reduce. A subsequent POC with Jericho3/Ramon3 silicon using 32 GPU cards showed even larger gains in multi-task scenarios: 26% improvement on multi-DataParallel AllReduce, 37% on multi-All2All, and 20% on mixed All2All+AllReduce workloads.

A major theme is ByteDance's push for standardization through the Open Scheduled Fabric (OSF) framework, which uses EVPN with extended community values to synchronize entries across endpoints, eliminating proprietary control plane protocols. OSF has been submitted as an IETF draft, and a multi-vendor agreement has been signed among ByteDance, H3C, DriveNets, Ruijie, Accton, and Broadcom. Prototype OSF devices were expected by end of October 2024.