Interconnect Wall

Overview

The interconnect wall describes the physical scaling limit where interconnect bandwidth — the capacity to move data between chips, boards, and racks — cannot scale as fast as compute due to fundamental physics constraints: signal integrity at high frequencies, power dissipation per lane, and package density limits. In AI systems, compute performance (FLOPs) scales with transistor count and GPU density, but the number of I/O lanes and lane speeds that can be practically routed on a substrate or across a backplane lags behind. This makes the interconnect wall, in the framing popularized by TE Connectivity, a more immediate bottleneck than the memory wall in current AI architectures.

The dynamic manifests in several ways: rack-to-rack bandwidth becomes a choke point as GPU counts per rack grow from 36 to 72 to 144 to 288 (Kyber architecture); chip-to-backplane connections require co-packaged copper (CPC) substrates routing 1,024+ differential pairs to avoid PCB bottlenecks; and data lane speeds must advance from 56G to 112G to 224G to 448G on a ~1.5-year cycle — faster than the prior ~3-year GPU generation cycle. HVDC power and liquid cooling are enablers, but without matching interconnect bandwidth, compute density gains stall.