96 AMD MI100 GPUs with PIM
Samsung has created the world’s first large-scale computing system that employs GPUs with built-in processing-in-memory (PIM) chips. According to Business Korea, these memory modules, which were loaded onto 96 AMD Instinct MI100 GPUs, increased AI training performance by 2.5x.
PIM is a new type of computer memory that can accelerate computationally complex workflows handled by processors like CPUs and GPUs. Each memory module, as the name implies, is capable of processing data on its own, reducing the amount of data that must travel between the memory and the processor.
Samsung first demonstrated PIM-modified GPUs in October, but it wasn’t until recently that 96 PIM-modified GPUs were combined in a cluster. When compared to standard video memory, these modified MI100 chips not only performed 2.5x better but also consumed 2.67x less power, significantly increasing the GPUs’ efficiency when running AI algorithms.
Samsung has been working on PIM for quite some time. In 2021, the company demonstrated several implementations involving various memory types such as DDR4, LPDDR5X, GDDR6, and HBM2. On a test program involving a Meta AI workload, Samsung saw a 1.8x increase in performance with a 42.6% reduction in power consumption and a 70% reduction in latency in LPDDR5 form. Even more impressive, these results came from a standard server system with no motherboard or CPU modifications (all that changed was a swap to PIM-enabled LPDDR5 DIMMs).
SK Hynix released its PIM modules earlier this year, so Samsung isn’t the only company developing PIM chips. According to preliminary testing, SK Hynix’s GDDR6-AiM (Accelerator in Memory) application accelerated AI processing by 16x while reducing power consumption by 80%. That’s significantly faster than Samsung’s modified MI100s, but we don’t know what SK Hynix tested with, so it’s not a direct comparison.
Regardless, PIM appears to be a powerful solution for accelerating AI-accelerated workflows. “As the head of the AI Research Center at Samsung Electronics Advanced Institute of Technology, I want to make Samsung a semiconductor company that uses AI better than any other company,” Choi Chang-kyu, vice president and head of the AI Research Center at Samsung Electronics Advanced Institute of Technology, told Business Korea.
