Alibaba HPN: A Data Center Network for Large Language Model Training

Author: Jiaqi Gao, Staff Engineer, Alibaba Cloud Collection: 2024 OCP Global Summit - Networking

Summary

Alibaba Cloud presents HPN 7.0, their seventh-generation High Performance Network architecture purpose-built for large language model training clusters. The core thesis is that naively scaling accelerator clusters does not yield exascale computing power -- transformer model sizes have grown 750x every two years while Moore's Law delivers only 2x over the same period, meaning network design becomes the critical bottleneck. HPN 7.0 is a two-tier, multi-rail, dual-plane, front/back-end topology that uses 51.2 Tbps single-ASIC switches and supports up to 130,000 accelerators. The design eliminates hash collisions and congestion within segments by isolating each accelerator rail to a dedicated first-tier switch.

The presentation details two pillars of exascale achievement: consistent high-performance interconnect (the topology itself) and computation-communication co-optimization (custom hardware and software stack). Alibaba developed in-house 51.2T switches, 2x400G optical modules, programmable NICs, the ACCL collective communication library, Solar multipath transport protocol, AliNOS white-box switch OS, Nimitz container networking, and the PAI management platform. In production, HPN 7.0 delivers 96% linear scaling efficiency, 98.22% cluster availability, 100% CCL performance improvement over DCN+, 14.4% training throughput increase, and continuous operation exceeding 3 weeks with minute-level failure recovery.