NVMe™ Interoperable Migration of PCIe™ virtual SSDs

At the 2025 OCP Global Summit, Microsoft's Lee Prewitt and NVMe technical-proposal author Mike Allison present interoperable live migration of PCIe virtual SSDs. Lee frames why hyperscalers need seamless, unobservable VM live migration—keeping Azure's Annual Interruption Rate low across device failures, maintenance, and load balancing—and the drive to eliminate paravirtualization overhead via a minimal hypervisor. Mike details the in-process NVMe technical proposal that extends the exported NVM subsystem from NVMe-oF to PCIe, giving the host control of controller/namespace identifiers and a static template so a virtual SSD can be reconstructed identically on a destination and migrated without the VM noticing.

Key Takeaways

Live migration is required to keep Azure's Annual Interruption Rate low; it handles device/predictive failure, node maintenance with sub-1-second IO block time for firmware updates, and—most importantly—load balancing of large and small VMs.
Paravirtualization (hypervisor fully emulating the NVMe device) wastes fast-SSD performance and consumes sellable CPU cores, so the goal is a minimal hypervisor out of both the IO path and admin-queue trapping to shrink the attack surface.
The work builds on prior NVMe proposals: TP4165 (used-LBA reporting and dirty tracking) and TP4159 (controller-state migration, presented two years earlier), plus an in-process QoS proposal to rate-limit noisy neighbors.
The new PCIe exported NVM subsystem extends an NVMe-oF concept to PCIe, virtualizing the SSD into exported controllers and namespaces and letting the host define identifiers (NQNs, namespace IDs) that are normally fixed by the SSD.
A unique template ID statically defines supported commands, log pages, and features for an exported subsystem and which final-state data is migrated, so an identical subsystem can be built on a destination even when its underlying controller/namespace numbers are already in use—a case TP4159 didn't cover.
Open gaps: migrated state currently travels in plaintext (an acknowledged hole), SIOV/confidential-compute (TDISP) support is still being defined, and data integrity relies on the device's 10^-18 bit error rate with host-to-host transfer left outside NVMe scope.