Signal Integrity in Immersion: Overcoming Challenges for High-Performance IT Platforms

Author: Andy Young, aDoas; Kai Wang, Intel Collection: 2023 Global Summit - Cooling Environments

Summary

This presentation addresses signal integrity (SI) challenges introduced by immersing air-designed IT hardware in dielectric immersion fluids — a topic that emerged in importance alongside the increasing data rates of modern interconnects. The workstream, active for approximately 6 months at time of presentation, focuses on measuring, characterizing, and mitigating the electromagnetic effects of immersion fluid dielectric properties on high-speed I/O channels.

The fundamental issue is that immersion fluids have a dielectric constant (Er) of 1.78–2.2 (versus 1.0 for air), which increases signal propagation delay, causes impedance drops on exposed conductors, reduces insertion loss bandwidth, and changes resonant behavior of cables and connectors. However, immersion also provides SI benefits: dramatically narrower operating temperature range (25–50°C vs 20–95°C in air cooling), more stable Equalization (EQ) tuning, lower PCB and package loss due to lower operating temperature, and elimination of humidity and dust.

Testing across DDR5 (4800/5600 MT/s), PCIe 5.0, UPI 4, Ethernet 56G PAM4, and sealed optical Ethernet shows that DDR5, PCIe, and Ethernet all demonstrate less than 2% eye diagram difference between air and immersion at time zero, and no change after 24 months of operation. The workstream is now aligning on the OCP Open Accelerator Infrastructure (OAI 2.0) platform — which includes the OCP Accelerator Module (OAM) standard for GPU nodes — as the common test vehicle for immersion SI characterization at 112G PAM4 speeds.