Architecting the Datacenter Requirements for Scaling AI Clusters

Author: Rolf Brink (Moderator, CEO Promersion, OCP Immersion Project Lead); Kelley Mullick (CEO Avayla, OCP Immersion Industry Liaison Team Chair); Mohan Kumar (Chief Architect & VP, Stealth AI Startup, Intel Fellow); Amy Short (Senior Officer Denvr Dataworks, OCP Immersion Solutions Committee Chair); Steve Mills (Mechanical Engineer & Technical Lead, Meta, OCP Cooling Environments SC Rep) Collection: 2025-OCP-EMEA-Summit-Immersion-Cooling

Summary

This panel discussion at OCP EMEA Summit 2025 (Dublin, 29–30 April) brought together five industry leaders to debate the practical engineering challenges of scaling AI clusters with liquid cooling. The central thesis: AI infrastructure has crossed a thermal inflection point where air cooling is no longer viable for high-density compute, and the industry must now choose among cold plate, immersion, two-phase, and hybrid approaches — all coexisting within the same data centers for decades to come. The panelists represented the full stack from chip architecture (Mohan Kumar, Intel Fellow at a stealth AI startup) through hyperscale operations (Steve Mills, Meta) to cloud service providers (Amy Short, Denvr Dataworks) and ecosystem consultancy (Kelley Mullick, Avayla).

The discussion surfaced three key tensions: (1) rack power density has escalated from 5–8 kW to hundreds of kW, with megawatt dual-rack configurations now imminent, requiring a complete redesign of data center thermal infrastructure; (2) existing large installed bases of air-cooled data centers must be bridged to liquid-cooled architectures using transitional technologies such as Meta's air-assisted liquid cooling (ALC); and (3) immersion cooling faces fundamental readiness gaps — no native immersion hardware, material compatibility issues, serviceability challenges, and absent industry standards — that must be resolved before large-scale deployment is feasible. The panelists converged on OCP-driven standardization as the critical near-term action, and challenged the community to deliver specifications for manifolds, quick disconnects, CDUs, and a unified management plane within six months.

Amy Short provided the most concrete operational data: Denvr Dataworks has run single-phase immersion for 2.5 years, achieving 40% power efficiency savings and significantly extended server lifetime through bulk fluid temperature stabilization. Mohan Kumar highlighted that thermal throttling is a real, frequent problem even in well-designed AI clusters due to pump-out effects and non-uniform die scaling, and that current servers are "air-cooled designs stuck in immersion" — leading to material incompatibility, fraying cables, optical refractive index issues, and serviceability nightmares. Steve Mills described Meta's Catalina platform (Nvidia GB200-based) as a flexible rack that can be deployed in either air-cooled or liquid-cooled data centers, and explained Meta's ALC (air-assisted liquid cooling) system as the bridge technology converting existing air-cooled facilities.