Search "HBM4" and the suggestions are "bandwidth", "stock" and "HBM4 vs HBM4e". Meanwhile the RAM in everything else keeps getting pricier. The two are connected. Here's what HBM is, in plain terms.
What HBM is
A normal RAM stick puts memory chips side by side on a board, a few centimetres from the processor, connected by a fairly narrow bus. High-bandwidth memory (HBM) does three things differently:
- It stacks the chips. Several DRAM dies sit on top of each other, with one base die at the bottom.
- It sits right next to the processor, on the same package.
- It uses a very wide connection, thousands of bits wide instead of 64.
The result is a huge amount of bandwidth (data moved per second) in a small, power-efficient package. The downside is cost: stacking, special packaging and lower yields make it far pricier per gigabyte than ordinary DRAM.
Why AI chips need it
When a large language model writes a reply, each new token means reading the model's weights from memory. For big models, the chip often spends more time waiting for data than doing math. So memory bandwidth sets a ceiling on how fast tokens come out.
Capacity matters too. Long context windows need a KV cache (the model's working memory for the conversation), and that cache lives in GPU memory. More HBM per GPU means longer contexts or more users per chip.
For comparison, NVIDIA's top gaming card, the RTX 5090, uses GDDR7 at 1,792 GB/s according to NVIDIA's spec page. A single HBM4 stack can reach 2 TB/s under the standard, and data-center GPUs carry several stacks.
What HBM4 changes
The JEDEC HBM4 standard (JESD270-4), published in April 2025:
| HBM4 (JEDEC) | |
|---|---|
| Interface width | 2,048 bits (double HBM3) |
| Speed | Up to 8 Gb/s per pin |
| Bandwidth per stack | Up to 2 TB/s |
| Channels | 32 (HBM3 had 16), each with 2 pseudo-channels |
| Stack heights | 4, 8, 12 or 16 high |
| Die density | 24Gb or 32Gb |
| Max capacity per stack | 64GB (16-high, 32Gb dies) |
| Compatibility | Controllers can also work with HBM3 |
| Reliability | Adds directed refresh management (DRFM) against row-hammer attacks |
Vendors can ship parts faster than the standard's top speed. For example, at 11 Gb/s per pin, a 2,048-bit interface works out to about 2.8 TB/s (2,048 × 11 ÷ 8 = 2,816 GB/s).
8-Hi vs 12-Hi: the 2027 trade-off
The "Hi" number is how many DRAM dies are in the stack. TrendForce's Sept 29, 2026 report says GPU and ASIC makers are considering 8-Hi stacks instead of 12-Hi for 2027. The reason isn't weak demand; it's limited HBM supply and fast-rising system costs. Fewer dies per stack means the same HBM output can supply more GPUs.
The catch: every stack needs one base die no matter how tall it is. With fewer DRAM layers to spread that cost over, TrendForce expects 8-Hi to cost roughly 10–20% more per gigabit than 12-Hi in 2027.
Why it affects your RAM
HBM and ordinary DRAM come out of the same advanced-process factories. TrendForce says HBM and conventional DRAM are "in competition for limited advanced-process and wafer capacity", and it expects memory supply to stay tight through 2027. It projects blended HBM prices up 121% year over year in 2027.
When suppliers can sell a wafer as expensive HBM, the DDR5 and LPDDR in PCs and phones gets squeezed. That's the main reason behind why RAM is so expensive right now.
The takeaway for builders
- Token speed is a memory problem. When you compare GPUs for inference, check memory bandwidth before compute numbers.
- Context costs memory. Long-context features in models like DeepSeek V4.1 Flash are largely about shrinking the KV cache, because HBM is scarce and expensive.
- Plan for tight supply. If you rent GPUs, expect high-memory instances to stay in demand. See our GPU rental price guide.
