AI Network Telemetry

Overview

AI Network Telemetry encompasses the monitoring and diagnostic approaches specifically designed for AI training cluster fabrics, where traditional telemetry methods (SNMP polling, coarse counters, In-band Network Telemetry) have proven fundamentally inadequate. AI training traffic is bimodal (full line rate or zero), globally synchronized, and extremely sensitive to the weakest link — a single degraded NIC out of 100,000 can cost an entire day of training time. Three distinct approaches emerged at the 2024 OCP Networking track: NVIDIA's symmetry-based histogram aggregation, Alibaba's Alternative Marking DSCP for end-to-end packet loss detection, and eBay's sFlow Drop Notification for vendor-neutral drop monitoring.

These telemetry approaches are tightly coupled to AI Traffic Engineering and Adaptive Routing For AI — telemetry feeds the real-time path quality signals that routing decisions depend on, and it is also the primary mechanism for detecting the fabric degradation events that cause disproportionate training performance collapse.