MangoBoost Full-Stack AI Infrastructure Solutions: MLPerf Inference, Training, Storage Case Studies

Author: Eric Vitadi, Chief Product Officer & Co-Founder, MangoBoost Collection: 2025 OCP Global Summit - Expo Hall Sessions

Summary

MangoBoost presents its full-stack AI infrastructure solution comprising three product lines: LLMBoost (full-stack AI software for inference, training, and fine-tuning with auto-tuning), GPUBoost (FPGA-based hardware-accelerated RoCEv2 RDMA NIC with programmable congestion control and UEC features), and BoostX (NVMe-over-Fabric accelerator for storage with full hardware TCP/RoCEv2/NVMe offload engines). All products are validated through peer-reviewed MLPerf closed-division submissions across inference, training, and storage benchmarks.

MLPerf Inference results show: first-ever multi-node scaling to 4 nodes with AMD MI300X GPUs using LLMBoost software (24% improved throughput versus similar H100 configurations); latest MLPerf 5.1 scales to 6 heterogeneous nodes (4x MI300X + 2x MI325X) with near-linear scaling, achieving best results in the submission table. MLPerf Training: first multi-node training with AMD MI300X GPUs across 4 nodes using GPUBoost RDMA NIC and LLMBoost software. MLPerf Storage: first hardware-accelerated NVMe-over-Fabric submission (both initiator and target), doubling utilization between v1 and v2 submissions, achieving highest performance with fewer nodes than software-only competitors. Autotuning capability is highlighted: vLLM has 144 tunable parameters; LLMBoost auto-tuning delivers 2-3x inference throughput improvement over untuned vLLM, and 2x faster training versus Megatron baseline. The GPUBoost FPGA-based DPU provides full hardware acceleration at line rate with software programmability (including custom congestion control), supporting UEC features like packet spraying. An AI virtual demo cloud is available for evaluation at MangoBoost's hosted infrastructure.