Alibaba's Ethernet Based DC Deployment for AI/ML Workloads Using Merchant Silicon

Author: Sasha, Alibaba (DC Network); Companion Presenter, Broadcom (Lead DC Product Manager) Collection: 2023 Global Summit - Networking

Summary

Alibaba and Broadcom jointly present Alibaba's approach to building large-scale AI/ML training cluster networks using Ethernet-based merchant silicon and open networking stacks. Alibaba's AliCloud network spans four major blocks: DCI network (SRv6-based backbone), Distributed Edge Cloud, DC Network (front-end services, storage), and HPN (High Performance Network for AI). All use white box switches with SONiC. Over six years, Alibaba has deployed three generations of HPN networks, evolving from 25G server/100G uplinks, to 100G downlinks/200G uplinks, to the current 400G HPN with 200G GPU connectivity and 400G spine links. Key innovations developed in production include HPCC congestion control (using in-band data plane information for better link utilization and lower latency) and Mirror-on-Drop/Mirror-on-Latency for rapid failure detection.

The presentation identifies three critical challenges for AI training networks at Alibaba's scale (10,000+ node clusters): very large scale requiring flatter topologies (3-stage Clos is hard to manage and balance), high performance demands for synchronous collective communication patterns with elephant flows, and extreme failure sensitivity where a single link failure can cause entire training jobs to checkpoint-rollback. Broadcom's solution centers on Tomahawk 5 (51.2T) switches enabling 6x switch reduction by flattening two-tier networks to single-tier (256 GPUs on one switch at 200G each), Dynamic Load Balancing using path quality metrics (queue size and port loading) instead of static 5-tuple ECMP, Global Load Balancing that extends congestion awareness one hop ahead, Reactive Path Rebalancing for probabilistic path reassignment, fast link failover under 500 nanoseconds, IFA in-band telemetry for queue depth visibility, and hardware-accelerated Mirror-on-Drop for automated root cause analysis.