LLM Inference Acceleration

Overview

LLM inference acceleration refers to hardware and systems optimized to run the forward pass (token generation) of large language models at scale, with high throughput and low latency. Unlike training workloads, inference is latency-sensitive, memory-bandwidth-bound (at the autoregressive decoding phase), and cost-sensitive — requirements that favor specialized AI accelerators over general-purpose GPUs.

The four-party AI CPU chiplet platform developed by Rebellions, Samsung Foundry, Arm, and ADTechnology targets LLM inference as its primary use case. By co-packaging REBEL_H NPU chiplets (high-performance low-power AI compute) with Arm Neoverse V3 CSS CPU chiplets (best-in-class server CPU) via UCIe die-to-die interconnects and Samsung Foundry's advanced packaging, the platform aims for approximately 3x power and performance efficiency versus existing LLM inference solutions. The combination of on-package CPU and NPU avoids the PCIe bottleneck between host CPU and discrete accelerator that limits conventional deployments.