SAI APM: Advanced Performance Monitoring

Author: Rita Hui, Principal Software Engineer, Microsoft Corporation; Jai Kumar, Distinguished Engineer, Broadcom Inc. Collection: 2023 Global Summit - Networking

Summary

Microsoft and Broadcom jointly presented a proposal for SAI Advanced Performance Monitoring (APM), a new framework for measuring and monitoring the performance of the SAI (Switch Abstraction Interface) adapter layer in production network deployments. The work is driven by Microsoft's expansion of SONiC into T2 (Tier-2) spine chassis switches, which manage over 300 clusters per chassis with multiple ASICs, multiple SAI instances, larger routing tables, more LAGs, larger ECMP groups, and significantly higher scalability requirements than T0/T1 devices. This scale demands the ability to quantify SAI programming performance across different silicon vendors and deployment scenarios.

The APM architecture introduces a new PerfMon (Performance Monitoring) object in the SAI hierarchy, attached as a child of the root switch object. Each PerfMon object specifies three attributes: the SAI object type to monitor, the specific API to observe (create, set, get, bulk create, etc.), and the performance metrics to gather. This design allows granular measurement -- for example, monitoring only bulk create operations on route objects. The initial proposal defines three core metrics: requests per time unit (throughput), average/instantaneous/max latency (covering combined SAI adapter + SDK + hardware time), and errors per time unit.

The proposal was in a private branch at the time of presentation (available at github.com/JaiOCP/SAI/tree/patch-4) and had not yet been raised as a community PR. The presenters called for community contributions to develop SAI/SDK benchmarks using PerfMon and to extend the metrics. The Q&A discussion highlighted the need for gNMI/OpenConfig integration to stream metrics and the question of whether APM should serve as pre-commit testing criteria or periodic operational monitoring.