Revolutionizing Networking for Gigawatt AI Factories

Author: Yuval Barenshtein, Spectrum X Product Manager, NVIDIA Collection: 2026-OCP-EMEA-Summit-Photonics

Summary

Yuval Barenshtein, Spectrum X Product Manager at NVIDIA, presents the full networking architecture for gigawatt-scale AI factories, structured as a "five-layer cake": energy → infrastructure → models → applications → AI agents. The talk covers the four distinct networks within an AI factory — scale-up (single-GPU formation from multiple ASICs), scale-out (hundreds of thousands to half a million GPUs in a single facility), scale-across (multi-site federation), and the newly introduced context memory storage network (KV cache movement for inference). The scale-out network is the primary focus of the presentation.

The centerpiece product is Spectrum-X, NVIDIA's purpose-built Ethernet networking solution for AI factories. Spectrum-X uses adaptive routing (per-packet, switch-mesh aware), SuperNIC-based congestion control (injection rate limiting before congestion hits), and hardware-software co-design with quarterly software updates. For the Nemotron ultra pre-training model, NCCL collective optimization achieves 2x improvement in Gradient All Reduce and Token Dispatch, 3.5x in Gradient Reduce Scatter, and 14x in Parallel Matrix Multiplication. Scale-out training performance shows 42% higher step-time consistency in multi-tenant data centers and 1.9x higher NCCL bandwidth between sites with Spectrum-XGS. Scale-across (multi-site) achieves nearly 100% effective bandwidth — twice that of off-the-shelf Ethernet — by using injection rate limiting instead of deep buffers.

The critical advance detailed is co-packaged optics (CPO) developed jointly with TSMC using a new "Co-op" process. Spectrum-6 Ethernet boxes use the industry-first 1.6 Tbps silicon photonics CPO chip with new Micro Ring Modulators (MRM) and 200G SerDes lanes. CPO reduces laser count by 4x versus pluggable data centers, delivers 3x more GPUs per unit of facility power (iso-power comparison), and achieves 5x higher AI workload uptime due to superior MTBF. The Spectrum-X Multi-Plane Multi-Rail topology supports up to 512K GPUs (Vera Rubin generation) with 1.6 Tb/s scale-out bandwidth per GPU, using 100T switches with 512 ports of 200G and 8 planes/4 rails; the base multi-rail configuration supports 8K Rubin GPUs with 100T switches at 64 ports of 1.6T.

NVIDIA DSX (Data center eXperience) provides reference designs and the DSX Air digital twin simulation platform, reducing infrastructure validation from 6 months to 1 week, software bring-up from months to 1 week, and deployment from weeks to 1 day via configuration copy-paste.