Although AI has attracted the attention of many, it had only been improving the quality of existing services until 2022 when Open AI announced ChatGPT, after which various LLM based AI services have been making a sensation. As LLM has become the mainstream in AI models, larger memory capacity and better memory performance have become necessary to process LLM service, leading to an era where high-performance memory, known as HBM, is now widely recognized by everyone. In this talk, I will introduce the trends in AI models and discuss HBM, the memory currently used for GPUs to process these AI models. Additionally, by using the chatGPT service as an example, I will demonstrate the limitation of the current GPU-based computing infrastructure and introduce SK hynix’s memory device. AiM, along with the AiMX, an LLM accelerator that overcome these limitations by utilizing processing in memory technology. Furthermore, I’d like to demonstrate the operation of an actual AI service, and present various memory and computing fusion solutions, as well as a comprehensive architecture that can be applied to the entire AI service data pipeline.

Eui-cheol Lim is a Research Fellow and leader of Solution Advanced Technology team in SK Hynix. He received the B.S. degree and the M.S. degree from Yonsei University, Seoul, Korea, in 1993 and 1995, and the Ph.D. degree from Sungkyunkwan University, suwon, Korea in 2006. Dr.Lim joined SK Hynix in 2016 as a system architect in memory system R&D. Before joining SK Hynix, he had been working as an SoC architect in Samsung Electronics and leading the architecture of most Exynos mobile SoC. His recent interesting points are memory and storage system architecture with new media memory and new memory solution such as CXL memory and Processing in Memory. In particular, he is proposing a new computing architecture based on PIM, which is more efficient and flexible than existing AI accelerators, to process generative AI and LLM (large language Model) that is currently causing a sensation.