1. What HBM is (the short version)

HBM, short for High Bandwidth Memory, is in one line DRAM stacked vertically, placed right next to the processor, and wired up with an absurd number of connections. JEDEC, the international standards body for semiconductors, published the first version in 2013.

• Stack it: four to sixteen DRAM dies piled on top of one another to form a single cube
• Move it closer: not out on the board but inside the same package as the processor, millimetres away
• Widen it: 1,024 data wires, and 2,048 in HBM4Sourced (JEDEC, 2025)

Put the three together and the HBM4 standard defines up to 2 TB/s from a single stack Sourced. One ordinary DDR5-6400 module in a PC delivers roughly 51.2 GB/sOur calculation. That is a gap of about forty times.

2. Why HBM had to exist

Compute got faster; the data stopped keeping up

A computer's performance rests on two things: how fast it can calculate, and how fast it can move data. Over the past few decades the first grew explosively while the second did not. The result is that arithmetic units finish their work and then sit waiting for the next batch of data. This is the problem known as the memory wall.

Generative AI, and large language model inference in particular, shows it in an extreme form. Every single token the model emits requires its weights to be read out of memory. Once a model runs to hundreds of gigabytes, throughput stops being a question of how fast you can compute and becomes a question of how fast you can read.

Raising the clock ran out of road

In principle there are only two ways to move more data: (A) run each wire faster, or (B) add more wires.

DDR and GDDR took route (A). But driving a trace that runs several centimetres across a printed circuit board at a high frequency degrades the signal and sends power consumption climbing. The faster each wire runs, the stronger both the transmitter and the receiver have to be, and both power and noise hit a wall.

The HBM answer: make the wires short, then win on count

HBM took route (B). Routing more than a thousand high-speed traces across a board is not physically practical, so HBM changed the premise instead.

• Stack the DRAM vertically so it takes up less area in plan view
• Put it inside the same package as the processor, cutting the wiring distance from centimetres to millimetres
• Connect through silicon wiring layers (an interposer) rather than a board, which allows thousands of traces at µm dimensions

Short wires reflect and attenuate far less, which means lower voltages and lower frequencies can still carry enormous traffic. The HBM3 standard does exactly that: a 0.4 V low-swing host interface with the supply dropped to 1.1 V Sourced (JEDEC, 2022).

3. How it differs from DDR and GDDR

The table scrolls sideways. The 192 GB/s figure for GDDR7 is the per-device ceiling set by JEDEC JESD239. A GPU board reaches its total by placing several devices side by side.

The point worth holding on to is that HBM is not a better DDR. You cannot add HBM capacity afterwards and it is expensive. DDR gives up bandwidth but is cheap, scales to large capacities and can be swapped out. Today's AI servers carry both, with each doing the job it is good at.

4. Where the bandwidth actually comes from

Memory bandwidth follows one very simple expression.

Look closely and the striking thing is that HBM is barely faster per wire than DDR5: 8 Gb/s for HBM4 against 6.4 GT/s for DDR5-6400. HBM wins not because each wire is quick but because there are thirty-two times as many of them. Not faster, wider — which is exactly what the word Bandwidth in the name is pointing at.

Fig. 2 · Schematic comparison of wire count and distance for DDR5, GDDR7 and HBM4

5. Structure: TSVs, microbumps, base die, interposer

HBM is not a single component. It is a structure assembled in three dimensions out of several elements. Here they are from the bottom up.

Fig. 4 · Conceptual cross-section of an HBM package

What each part does

• DRAM core dies: the layers that actually hold the data. The JEDEC standards define 4-, 8-, 12- and 16-high configurationsSourced.
• Base die: the control chip at the bottom of the stack. It handles the external interface and the test functions.
• TSV (through-silicon via): a copper electrode that passes clean through the silicon die. Without it there is no short path between stacked chips, which makes it the element that lets HBM exist at all.
• Microbumps: the tiny joints between die and die, and between die and interposer. They are copper pillars topped with solder.
• Encapsulant: the resin filling the gaps between dies. It protects the stack mechanically and also serves as a path for heat (see section 6).
• Silicon interposer: the floor that carries both the HBM stacks and the logic die. Fine wiring on silicon is what makes thousands of connections possible.

6. A materials engineer's view: thinning, encapsulation, warpage, heat

The race in HBM is, in truth, as much a race in materials and process as it is a race in DRAM circuitry. This is the part that most explanations of HBM leave out.

(1) Thinning the dies, because the height budget will not move

The total package height of HBM is fixed, so the only way to add layers is to make the DRAM chips themselves thinner. SK hynix has stated that its 12-high HBM3 uses DRAM chips 40% thinner than the 8-high partSourced (SK hynix). Samsung says its TC-NCF technology fits a 12-high part into the same package height as an 8-high oneSourced (Samsung, 2024).

Thinner silicon, naturally, warps more easily and cracks more easily. From here on, everything is a question of materials and stress.

(2) Two schools of encapsulation: MR-MUF and TC-NCF

How do you fill the gaps between stacked dies? Two broad approaches exist, and manufacturers have split between them.

(3) Keeping warpage under control

SK hynix describes a chip control technique in which, at each stacking step, a brief burst of heat fuses the bumps of the upper chip to the pads of the lower one, so that those pads hold the chip in place and prevent it from warpingSourced (SK hynix). Thinning, stacking, encapsulation and warpage control are not separate problems. They are one set of simultaneous equations whose terms trade against each other.

(4) The floor itself may change: from silicon to organic substrate

HBM has always assumed a silicon interposer underneath. In June 2026, however, JEDEC published SPHBM4 (JESD330-4)Sourced. It uses the same DRAM dies as HBM4 but routes them through a new interface base die so that the stack can be mounted on an ordinary organic substrate.

The mechanism is straightforward. Where HBM4 uses 2,048 data signals, SPHBM4 reaches comparable total throughput with 512 signals and 4:1 serialisationSourced. Cutting the signal count to a quarter relaxes the bump pitch, and a relaxed pitch is what makes an organic substrate viable (JEDEC).

7. Generation by generation, from HBM to HBM4

(A) How the JEDEC standards moved

Every row is Sourced from JEDEC's published material. JESD270-4 was revised to JESD270-4A (Version 1.1) in December 2025.

(B) Products and deployments

All rows are Sourced from the companies' own material, listed at the end. Only the 128 GB/s per stack figure in the AMD row is a calculation by this article.

8. Why it decides what an AI accelerator can do

Inference speed is a memory bandwidth question

To emit one token, an LLM must read out the model weights. At small batch sizes the amount of data read per arithmetic operation is large, so the ceiling on throughput is memory bandwidth, not the arithmetic units. Double the bandwidth in that regime and throughput moves roughly in step.

Capacity matters just as much

If the model does not fit in memory, it has to be split across several GPUs that then talk to each other, and that communication becomes the new bottleneck. HBM is one of the few ways to buy bandwidth and capacity at the same time.

What the shipping hardware shows

Both are NVIDIA's published figuresSourced. NVIDIA describes Rubin as delivering 2.8 times the bandwidth of Blackwell and Blackwell Ultra.

Divide Rubin's 22 TB/s across its 12-high stacks and you get about 2.75 TB/s per stackOur calculation, which lines up well with the above 2.8 TB/s that Micron publishes for HBM4. It confirms that an AI accelerator's bandwidth is essentially one stack's performance multiplied by the number of stacks.

9. What is still hard: heat, yield, cost

(1) Heat

A die in the interior of a stack is sandwiched between other dies and has few routes out for its heat. More layers, thinner dies and faster operation all make this worse. The reason every manufacturer competes on encapsulant thermal conductivity and thermal dummy bump counts is that this constraint sets the real ceiling on performance.

(2) Yield

By construction, a single bad die makes the whole stack bad. A 12-high part becomes a product only if all twelve dies are good and all eleven bonding interfaces succeed. The more layers you add, the worse the final good-part rate becomes. That follows from the structure itself (this paragraph argues from structure and makes no claim about any manufacturer's yield figures).

(3) Assembly cost

HBM presupposes advanced packaging such as a silicon interposer. In TSMC's CoWoS, interposer area has grown from roughly one reticle, about 830 mm², to the 3.3-reticle classSourced (TSMC). The more large-area silicon you use, the heavier the yield and cost burden, so the cost driver is not the memory alone but the whole packaging flow. Read that way, SPHBM4 looks like the standards world's answer to a structural cost problem.

10. What comes next

11. Glossary

12. Primary sources

1. JEDEC "JEDEC and Industry Leaders Collaborate to Release JESD270-4 HBM4 Standard", 16 April 2025 — jedec.org
2. JEDEC "High Bandwidth Memory (HBM4) DRAM / JESD270-4A", December 2025 — jedec.org
3. JEDEC "JEDEC Publishes HBM3 Update to High Bandwidth Memory (HBM) Standard", 27 January 2022 — jedec.org
4. JEDEC "JEDEC Updates Groundbreaking High Bandwidth Memory (HBM) Standard" (JESD235A), 12 January 2016 — jedec.org
5. JEDEC "JEDEC Updates Groundbreaking High Bandwidth Memory (HBM) Standard" (JESD235B), 17 December 2018 — jedec.org
6. JEDEC "JEDEC Prepares SPHBM4 Standard to Deliver HBM4-Level Throughput with Reduced Pin Count", 11 December 2025 — jedec.org
7. JEDEC "Main Memory: DDR SDRAM, HBM" (records the June 2026 publication of JESD330-4 and the HBM5 work in progress) — jedec.org
8. JEDEC "JEDEC Publishes GDDR7 Graphics Memory Standard" (JESD239), 5 March 2024 — jedec.org
9. AMD "AMD Ushers in a New Era of PC Gaming with Radeon R9 and R7 300 Series", 16 June 2015 — ir.amd.com
10. SK hynix "SK hynix Starts Mass-Production of High-Speed DRAM, HBM2E" — news.skhynix.com
11. SK hynix "SK hynix Completes World's First HBM4 Development and Readies Mass Production", 12 September 2025 — news.skhynix.com
12. SK hynix "Rulebreakers' Revolutions: MR-MUF Unlocks HBM Heat Control" — news.skhynix.com
13. Samsung "Samsung Develops Industry-First 36GB HBM3E 12H DRAM", 27 February 2024 — news.samsung.com
14. Micron "HBM3E" product page — micron.com
15. Micron "HBM4" product page — micron.com
16. NVIDIA "H200 Tensor Core GPU" — nvidia.com
17. NVIDIA "Inside NVIDIA Rubin GPU Architecture" — developer.nvidia.com
18. TSMC "CoWoS" technology page — 3dfabric.tsmc.com

13. Claim-to-source audit

Every significant number in this article, with where it appears, what it rests on and how it is labelled.

Last updated 20 September 2026. Sources are limited to primary material: JEDEC's published standards documentation, official manufacturer announcements and product pages. All figures are explanatory concept graphics. Visual-overview figures are shown as AI-generated conceptual images, matching the Japanese edition. Vector drawings are retained where they carry quantitative values or precision-critical technical labels. AI-generated images do not depict real equipment, products, facilities, dimensions or exact cross-sections; replaced source SVG overview drawings remain in the HTML but are hidden.