Best Practices for Liquid & Air Cooling of a 51.2Tbps Switch for High-Density AI Clusters

Author: Henry Wu, Switch Product Marketing, Broadcom; Fangbo (Fu), Senior Network Architect, Alibaba Collection: 2024 OCP Global Summit - Networking

Summary

Broadcom and Alibaba present a joint perspective on cooling 51.2 Tbps switches for high-density AI clusters, covering both silicon-level power optimization and system-level thermal design for air and liquid cooling. Data center rack power density has increased roughly 10x in five years, from 12 kW/rack (ORv2 in 2017) to 92-140 kW/rack (ORv3 High Power Rack in 2022), driven by AI computation demand. Broadcom argues that monolithic 51.2T switch integration (Tomahawk 5) displaces six 25.6T switches while saving at least 75% silicon power, and that each silicon generation achieves approximately 30% power efficiency improvement through process advancement over a trajectory from 640G Memory (2010) to 51.2T Tomahawk 5 (2022) -- an 80x bandwidth increase with 95% power efficiency improvement.

Alibaba's Fangbo presents their production deployment experience with Tomahawk 5 in HPN 7.0 AI clusters. The Tomahawk 5 system reduces network cluster power by 35% compared to Tomahawk 4 systems connecting the same number of GPUs. Alibaba designed both air-cooled (4RU, 128x QSFP112, max 35C ambient) and liquid-cooled variants. The air-cooled design uses only two PCB layers and five system fans with vapor chamber heat sinks, saving millions of dollars by targeting 35C rather than 40-45C ambient -- a standard Alibaba has operated successfully for four years. The liquid cooling solution uses PG25 fluid carrying 80% of system heat, with aluminum cold plates embedded with copper tubes covering stacked QSFP112 optical modules. Liquid cooling saves approximately 800W at peak ambient temperature, enough to power an additional GPU card per switch. The liquid cooling cost premium over air cooling is less than 10%, driven by smaller fans and PSUs offsetting the cold plate cost.