Processing-in-Memory

Overview

Processing-in-Memory (PIM) is a memory architecture that places arithmetic logic units (ALUs) adjacent to DRAM bank arrays, enabling computation to execute directly where data resides rather than shuttling it across the memory bus to a remote CPU or GPU. In a conventional memory architecture, only one bank within a rank is active at a time; with PIM, instructions execute in parallel across all banks in a rank simultaneously, dramatically increasing effective bandwidth and reducing the energy cost of data movement.

PIM is particularly well-suited to workloads with low arithmetic intensity — specifically, attention layers in large language models — where the bottleneck is data movement rather than floating-point compute throughput. Samsung's LPDDR5X-PIM implementation demonstrated 4.47x system performance and 3.44x energy efficiency improvement over a standard NPU+LPDDR5 configuration for on-device generative AI inference.