Firefly

Overview

Firefly is Microsoft's software mitigation for AI training data center power stabilization. It addresses the near-idle power drop during the network-communication phase of AI training (when GPUs are waiting for collective operations like AllReduce to complete) by launching a secondary power-hungry workload on the same GPUs during those low-power windows.

Using NVIDIA MPS (Multi-Process Service) for context switching, Firefly achieves below 5% performance impact on the primary training job while ramping the secondary workload up to 100% TDP — reducing the overall power swing to below 5% (excluding EDP transitions). This meets the tightest utility grid frequency-domain specifications for power stability.

The control logic uses millisecond-level GPU activity counters (obtained through collaboration with NVIDIA, whose standard telemetry is at tens-of-milliseconds granularity) to determine when and how aggressively to ramp the secondary workload. Initially the secondary workload is artificial; in future iterations it is expected to be low-priority productive work. Firefly is designed to complement NVIDIA's GPU Min Power Floor hardware feature: MPF handles the ±20% swing band, while Firefly is deployed when a data center operator needs to meet sub-20% specifications.