Energy-Efficient Optical Interconnects for Multi-Rack AI Pods

Author: Fotini Karinou, Principal Hardware Architect – Optical Technologies, Systems Planning and Architecture (SPARC), Azure Hardware Systems and Infrastructure (AHSI), Microsoft Collection: 2026 OCP EMEA Summit - Photonics

Summary

Fotini Karinou from Microsoft's Azure Hardware Systems and Infrastructure organization presented the hyperscaler's perspective on optical interconnects for AI scale-up networking. Microsoft builds across the full stack — from custom silicon (Cobalt 200 Arm CPU, Maia 200 AI accelerator for inference) through platform integration to full rack-scale systems. The context is extraordinary: for its "AI One" cluster, Microsoft deployed 120,000 new fiber miles across the US (enough to wrap the Earth 4.5 times), plus AI datacenter campuses in Wisconsin and Atlanta ("Fairwater"). This massive scale-out fiber build is described as the foundation of current distributed AI — and the next frontier is optics penetrating the scale-up domain, interconnecting tightly coupled accelerators across trays, racks, and even rows.

The talk laid out a quantitative performance specification for scale-up optical interconnects, derived from two driving applications: (1) scale-up communication (between accelerators) and (2) off-package memory interfaces. The combined target is: reach >20 m chip-to-chip; bandwidth per AI SoC >100 Tbps/direction; round-trip latency <500 ns; ASIC bandwidth density >4 Tbps/mm (Tx + Rx); energy efficiency <4 pJ/bit; BER (pre-FEC) <10⁻¹². These targets require optics to be strictly better than copper in every dimension. The presentation mapped the industry landscape across the pluggable → on-board optics (OBO) → co-packaged optics (CPO) spectrum, with CPO further subdivided into three sub-variants: a SerDes-based MCM approach (mainstream but high power), a multi-chip module CPO (good BW/power trade-off, established packaging), and a die-to-die interposer CPO (best BW density and power, but highest serviceability and reliability risk). Microsoft described a three-phase evaluation framework: Phase 1 (ingredient readiness: technology prioritization, integration complexity, link performance/power), Phase 2 (system design readiness: end-to-end link performance, reliability, accelerator/switch/rack design, fiber management, supply chain), and Phase 3 (deployment readiness: DC installation, RAS, telemetry, firmware, control plane, data plane). Each generation spans 2–3 years.

The keynote conclusion was a call to action around standardization. Microsoft co-founded the OCI-MSA (Optical Compute Interface Multi-Source Agreement) on March 12, 2026, with AMD, Broadcom, NVIDIA, Meta, and OpenAI. The OCI-MSA's mission: "Move Scale-up from In-Rack to Multi-Rack & Multi-Row" via an open specification for optical scale-up. Gen 1 of OCI-MSA covers NRZ, WDM, and BiDi architectures. Gen 2 is in planning. The speaker emphasized that standards must cover the full optically interconnected system stack — from materials through lasers, modulators, photodetectors, optical interfaces, electrical interfaces, form factors, packaging, connectors, fiber shuffles, rack/pod integration, and performance testing — and called for OCP to take ownership of specific stack layers not addressed by the OCI-MSA.