Edge AI Cluster

Overview

An edge AI cluster is a rack-based, on-premises computing environment designed for real-time AI inference, simulation, and closed-loop model refinement at deployment sites where cloud connectivity is unavailable or impractical. The canonical reference architecture includes IGX industrial edge compute nodes, OVX GPU servers for training and simulation (with dynamic resource allocation between the two), a storage node, a management/base command node, and high-bandwidth switches connecting to physical sensors via Holoscan Sensor Bridge with Ethernet GPU-Direct. Kubernetes orchestration manages workloads across the cluster nodes.

Edge AI clusters are essential for use cases such as factory floors, hospital operating rooms, and moving vehicles (trains, trucks) where data privacy regulations, latency constraints, or lack of cloud connectivity preclude offloading compute to the cloud. The on-premises model supports hardware-in-the-loop and software-in-the-loop testing alongside physical robot hardware, enabling rapid model iteration without round-tripping data to a remote data center.

Inventec's proof-of-concept implementation uses Inventec Edge IGX Orin systems for industrial-grade edge compute (robust security and safety tailored for advanced robotics), Inventec P90G6 OVX servers for simulation, and an OCP DC-MHS compliant EdgePro management node (Intel Xeon 6). The cluster scales in multiples of three from a minimal two-rack MVP layout.