SONiC - NOS of Choice for AI Data Centre

Author: Prosenjit Pal, VP AI Networking Software; RamanaMohanRao Dasari, Director AI Networking Software — Upscale AI Collection: 2026-OCP-EMEA-Summit-Executive-Sessions

Summary

Presented jointly by two executives from Upscale AI, this session argues that SONiC (Software for Open Networking in the Cloud) is becoming the de facto Network Operating System for AI data center fabrics — for both scale-up (intra-rack GPU-to-GPU) and scale-out (inter-rack) networking. The structural driver is the widespread adoption of Mixture of Experts (MoE) architecture in frontier AI models, which requires near-instantaneous all-to-all communication between GPUs and high-bandwidth memory (HBM). This communication pattern demands scale-up networking characterized by ultra-low latency and ultra-high bandwidth — distinct from traditional scale-out architectures — and drives SONiC adoption because of its distributed-by-design architecture, hardware/software abstraction layer, and unified provisioning model.

Market projections are cited to establish urgency: by 2030, scale-up architectures will capture nearly 60% of the total AI backend switch market (from a Dell'Oro Group report), and SONiC-based data center switch sales in scale-out networks are projected to grow at a 5-year CAGR exceeding 50%. The second demand wave — "Wave 2 Greenfield" — is new clouds and large enterprises building out AI data centers where heterogeneous compute, heterogeneous accelerators (different chips for prefill vs. decode inference), and multi-vendor storage are requirements. SONiC addresses heterogeneity through hardware abstraction, making it the only practical choice for vendor-neutral AI fabric orchestration.

Upscale AI (founded Q3 2025, $300M raised: $100M seed + $200M Series A, 200+ employees, 4+ generations of ASIC and system design shipped) is building purpose-built networking systems: the SkyHammer architecture for scale-up using open standards from the ground up, and a scale-out solution in strategic partnership with Nvidia's Spectrum-X Ethernet silicon.