The dominant HBM supplier is developing a new generation of technologies to tackle bottlenecks that bandwidth alone cannot resolve as increasingly sophisticated AI agents push data traffic to new extremes, Ahn Hyun, president and chief development officer of SK hynix, told SEDEX 2026 in Seoul.
Ahn outlined a strategy to improve the entire memory hierarchy, from three-dimensional stacked DRAM and high-bandwidth flash to shared memory systems and next-generation storage.
The shift comes as AI moves beyond generating answers to performing complex tasks through autonomous agents, driving a surge in computing and memory demand.
Ahn said AI inference — the process of running trained models — is expanding along three dimensions: task complexity, context length and the volume of requests.
“These three factors multiply rather than add up,” he said, describing their combined impact on consumption of tokens, the units of data AI models process.
He projected that AI infrastructure could expand nearly 20-fold by 2030 from 2026 as adoption accelerates, and the staggering expansion will likely shift the bottlenecks.
Traditionally, the “memory wall” referred largely to the speed gap between processors and memory. As AI models handle longer conversations and more complex tasks, moving data between HBM, conventional DRAM and storage is becoming a major constraint.
“The bottleneck ultimately comes down to data movement,” Ahn said.
Longer context windows — the amount of information a model can consider at once — require larger caches of previously computed data, known as key-value or KV caches, to avoid repeating calculations. As those caches outgrow the fastest memory, data must move across different memory and storage layers, adding delays and consuming more power.
Another is high-bandwidth flash, or HBF, which applies stacking and high-speed data-transfer technologies used in HBM to NAND flash memory.
SK hynix aims to achieve roughly 10 times the capacity of a comparable generation of HBM at about half its bandwidth, potentially allowing AI systems to keep larger datasets close to their processors.
Ahn said HBF could be particularly suited to inference, which repeatedly reads large volumes of data while writing relatively little.
The company is also developing memory pooling based on Compute Express Link, or CXL, a connection technology that would allow servers to share memory resources. Alongside it is Storage-Next, a high-performance architecture for enterprise solid-state drives.
Together, the technologies would add layers between existing memory and storage products, giving AI systems more options to balance speed, capacity and energy use.
The strategy does not signal a retreat from HBM, the high-margin product that has cemented SK hynix’s position in AI memory.
The company plans to advance its lineup through HBM4E and custom HBM, incorporating customer-specific logic and memory controllers into the base die — the bottom layer of the stack — to better match individual AI processor designs.
It is also working with partners on standards and commercialization. Its HBF efforts include collaboration with SanDisk and engagement with cloud service providers, processor developers and the Open Compute Project ecosystem.
Other SEDEX keynote speakers echoed the need to rethink semiconductor design as AI workloads become more demanding.
Seung-bae Park, a professor at Binghamton University, highlighted advanced packaging and the integration of different types of chips as ways to overcome hardware limitations. Shin Dong-joo, chief executive of South Korean AI chip startup Mobilint, emphasized energy-efficient neural processing units capable of handling real-time inference as AI moves into robotics and other physical applications.
Copyright ⓒ Aju Press All rights reserved.