SEOUL, September 22 (AJP) - South Korean memory makers racing to sustain a comfortable lead on the next-generation chip front have their eyes on the next big thing; high-bandwidth flash, or HBF.
Six months ago, one prominent chip researcher predicted it would define the AI path.
"HBM defined the last decade, but HBF will likely determine the next," Joung-ho Kim, a professor at the Korea Advanced Institute of Science and Technology (KAIST), told AJP in March.
The proposition is now acquiring technical specifications — and facing closer scrutiny.
SK hynix and Sandisk released the first open HBF specification in August, following the launch of their standardization consortium in February. Google and AI chipmaker Tenstorrent have joined the initiative, giving the technology backing beyond its memory suppliers.
But recent research suggests that faster flash alone will not deliver the gains its proponents envision. HBF’s success could depend as much on how AI systems move and manage data as on the performance of the chips themselves.
For Korean manufacturers, the challenge is to turn their memory expertise into another major AI business. For customers, the test is more practical: whether HBF can expand capacity and reduce costs without introducing new bottlenecks.
HBF is intended to fill the gap between high-bandwidth memory, or HBM, and conventional solid-state drives.
HBM uses stacked DRAM chips to feed processors data at extremely high speeds, but its capacity is expensive. SSDs use NAND flash to offer much larger storage volumes at lower cost, but cannot supply data at comparable speeds.
HBF aims to bring flash capacity closer to the processor with a much wider data connection. It would complement HBM, adding another tier between high-speed working memory and conventional storage.
The need is growing as AI companies expand inference — running trained models to answer questions and complete tasks.
Longer conversations, more elaborate reasoning and AI agents performing multistep tasks increase the amount of data systems must retain. Alongside model weights, the parameters learned during training, systems hold key-value, or KV, caches that preserve information from earlier parts of a sequence.
Keeping everything in HBM is costly. Moving too much to slower storage can leave expensive processors waiting for data.
The SK hynix-Sandisk specification sets out one proposed bridge between the two. It provides for stacks of eight or 16 NAND dies with capacities of up to 512 gigabytes and three bandwidth grades ranging from about 0.4 terabytes per second to 3 TB/s.
It also adopts Universal Chiplet Interconnect Express, or UCIe, an open interface intended to connect HBF with different processor types, including GPUs and CPUs. Released through the Open Compute Project, the specification covers electrical characteristics, packaging, reliability and software input-output requirements as well as capacity and speed.
A common specification gives chipmakers and system designers a basis for development. It does not establish how much performance finished systems will deliver.
NAND presents difficulties that DRAM does not share in the same way. Writing data is slower than reading it, and flash cells can withstand only a finite number of erase-and-write cycles. Workloads that continually rewrite data can therefore create endurance problems as well as performance and thermal constraints.
Those trade-offs are emerging in research examining where HBF could help — and where it could disappoint.
A study co-authored by Jisung Park, a professor at Pohang University of Science and Technology, or POSTECH, examined HBF across different configurations for serving large language models.
Published in IEEE Computer Architecture Letters, “Exploring High-Bandwidth Flash for Modern LLM Inference: Opportunities and Challenges” found potential for larger batches of requests, higher throughput and a reduction in the minimum number of GPUs required.
The benefits, however, depended on demanding assumptions.
"Realizing these benefits critically depends on sustaining HBM-comparable read bandwidth and requires significant endurance improvements," the researchers wrote.
The distinction is between peak speed and usable performance. A flash stack that briefly reaches a bandwidth target may still struggle to sustain it under continuous AI workloads.
A separate research team examined a more specific proposition: using HBF as a replacement for SSDs in an existing system that moves KV-cache data out of faster memory.
Its paper, “HBF Sucks? A Full-Stack Characterization of High-Bandwidth Flash for KV-Centric LLM Serving,” was first submitted to arXiv in August and revised in September. It remains a preprint rather than a peer-reviewed journal study.
Using an extended TokenSim framework, production workload traces and Nvidia H100 and B200 accelerator profiles, the researchers modeled what would happen when HBF was inserted into that architecture.
Average end-to-end latency rose to between two and 5.5 times the baseline. The maximum rate of work completed within specified service targets also fell.
"The device is fine; the drop-in deployment is not," they wrote.
Frequently reused data stayed in faster memory, leaving HBF with a write-heavy workload. The modeled stack encountered thermal limits below peak bandwidth and raised endurance concerns.
The authors favored selective use, with data placement based on likely reuse, limits on writes and coordinated thermal management. Their findings concern a particular simulated deployment, not a verdict on all HBF designs or measurements from commercial HBF chips.
The two studies approach different uses of the technology, but together point to a central issue: where data resides can matter as much as how quickly a chip can move it.
Frequently accessed information may need to remain in HBM. Data suited to repeated reading could make better use of HBF’s capacity. Rapidly changing caches require closer scrutiny because repeated writes can erode the advantages of faster flash.
The engineering work therefore extends beyond stacking NAND dies. Memory suppliers, processor designers and software developers must decide together which workloads justify HBF and how to keep them within its operating limits.
Kim anticipated that broader requirement in his March interview.
"The memory architecture must be fundamentally restructured," he told AJP.
Six months later, the research is giving that statement a more concrete meaning. Adding another memory tier also requires changing the decisions that govern it.
Other approaches are developing alongside HBF.
At FMS 2026 in August, Samsung Electronics displayed a concept model of zNAND-O, a high-performance NAND architecture under development in four- and eight-layer versions. Samsung said it was intended to improve data-transfer performance and reduce latency for real-time, data-intensive edge AI applications.
The company also showed zHBM, a concept that places HBM directly above an AI accelerator rather than alongside it, shortening the distance data must travel. Both were presented as architectures under development, not finished commercial products.
The competing designs suggest that the next stage of AI memory will involve several approaches to the same problem: supplying processors with enough data, quickly enough, at an acceptable cost and power budget.
HBF now has an open specification, industry participants and a growing body of research identifying its possible uses and limits. Whether it becomes a business on the scale of HBM remains unproven.
HBM demonstrated how valuable memory can become when it removes an AI bottleneck. HBF’s challenge is to do the same without creating another one.
AJP Takeaways
- SK hynix and Sandisk are pushing HBF as a new AI memory tier between high-speed HBM and lower-cost, high-capacity storage.
- Recent studies show HBF's potential depends on overcoming NAND endurance, thermal and write-performance limits, as well as optimizing how AI systems place data across memory tiers.
- HBF now has an open specification and growing industry backing, but whether it can develop into a market on the scale of HBM remains unproven.
Copyright ⓒ Aju Press All rights reserved.