The Automation Core

HBM4 16-High Stacking Explained: Why Bonding and Heat, Not Bandwidth, Set the Limit

· 16-high HBM, 775 um stack height, CoWoS, HBM4, high bandwidth memory, hybrid bonding, JEDEC JESD270-4, MR-MUF, NVIDIA Vera Rubin, package thermal resistance, Samsung HCB, SK hynix, TC-NCF, TSV,

Bottom line: In our video on exploding RAM prices, we showed how HBM soaks up DRAM wafer capacity. HBM4 pushes the problem one layer deeper. The interface spec doubled, but what actually gates shipments now is stack height, bonding yield and heat removal. Two weeks of October news lay this out clearly: NVIDIA Rubin will likely run mostly on 12-high stacks, and 16-high waits on yield and packaging capacity. Four engineering points follow. Any derived figures are my own illustrative arithmetic, not vendor data.

What happened

  • Oct 9, Korea Herald (citing Counterpoint): most Vera Rubin products are expected to use 12-high, 36 GB HBM4, not the rumored 8-high. SK hynix showed a 16-high 48 GB part at CES in January and began mass supply of HBM4 in Q2; Samsung has shipped HBM4 since February (12-high, up to 36 GB). A supply-chain tracker says SK hynix is shipping 16-high for Rubin, which the company has not confirmed.
  • Oct 12, TrendForce (citing Chosun Biz and ETNews): AI interposers are already about 5.5× the 858 mm² single-exposure EUV reticle limit and could reach 9×, 12× or even 40×. Samsung is preparing hybrid copper bonding (HCB) and a heat path block (HPB) for HBM4E; SK hynix says its iHBM cuts thermal resistance by about 30%.
  • Oct 16, vlsi.kr analysis: as of October, volume HBM4 still uses microbumps: Samsung’s TC-NCF and SK hynix’s MR-MUF. Hybrid bonding is expected in select HBM4E products around 2027 at the earliest.
  • Oct 15, SEMICON West (The Elec, Oct 19): Korean equipment maker Shindo proposed stacking the GPU directly above the HBM, separated by a roughly 200 µm glass-ceramic interposer, with liquid cooling on both faces. It has sampled the substrate to some companies.

How an HBM stack is built: several DRAM dies are stacked vertically on a logic base die, with through-silicon vias (TSVs) carrying signals and power through each layer, joined by microbumps or by hybrid bonding. The stack sits next to the GPU on a silicon interposer (the 2.5D arrangement of TSMC CoWoS), and a cold plate presses down on top of both.

Engineering point 1: double the bandwidth, double the wires

JEDEC’s HBM4 standard (JESD270-4) widens the interface from 1,024 to 2,048 bits at up to 8 Gb/s per pin, so 2,048 × 8 Gb/s ≈ 2 TB/s, and doubles the channel count from 16 to 32. Shipping parts already run faster: Samsung quotes 3.3 TB/s per stack on a 4 nm logic base die, which works out to about 12.9 Gb/s per pin (my arithmetic). Capacity is simple multiplication: twelve 24 Gb dies give 36 GB, sixteen give 48 GB, and the spec ceiling is 64 GB with sixteen 32 Gb dies. Twice the pins means far more connections that must align and conduct on every layer, and that is where bonding yield starts to hurt.

Engineering point 2: sixteen DRAM dies in 775 µm

JEDEC relaxed the HBM4 package height to 775 µm (HBM3E was 720 µm) so the cold plate can sit flat on both the GPU and the HBM. With the height fixed, more layers means thinner dies: vlsi.kr puts them at about 50 µm for 12-high and about 30 µm for 16-high. Do the sums: 12 × 50 = 600 µm, 16 × 30 = 480 µm. The 16-high stack actually carries less total silicon, because the four extra bond layers also have to fit in the height budget (my arithmetic). At 30 µm, warpage during thermal cycling is much harder to control, and one bad joint scraps the whole stack. That is why Rubin leans on 12-high first: 16-high can be built, but yield and cost are not there yet. Industry analysis also reports JEDEC discussions about raising the next height target to 825–900 µm (a proposal, unconfirmed), which would buy conventional bumps another generation.

Engineering point 3: heat has to cross sixteen insulating layers

The DRAM is not the only heat source. Samsung, as cited by TrendForce, identifies the D2D PHY in the base die, which drives the high-speed link to the GPU, as a major hotspot. That heat must climb through every silicon layer and every bond layer. Silicon conducts well (about 150 W/m·K), but the polymer underfill or NCF film between bumps is typically below 1 W/m·K (textbook order of magnitude), so each bond layer acts like a thin blanket, and more layers add up. There are two fixes. Remove the interface: hybrid bonding joins copper pad to copper pad with no bump gap, and Samsung claims HCB lowers thermal resistance by more than 20% versus conventional thermocompression (its own comparison). Give the hotspot its own exit: Samsung’s HPB and SK hynix’s iHBM both wrap the D2D PHY in a thermally conductive, electrically insulating path, and SK hynix estimates about 30% lower thermal resistance.

Engineering point 4: the bottleneck moves to the package

Fix the stack and the package is still the bigger problem. An interposer at 5.5× the 858 mm² reticle is roughly 4,700 mm² (my arithmetic). TSMC lists CoWoS-S interposers up to about 3.3× reticle (around 2,700 mm²), with CoWoS-L or CoWoS-R for anything larger. Bigger interposers mean more warpage, lower yield and tighter capacity, and the industry source quoted by Korea Herald says the same thing: memory output only matters if TSMC packaging capacity keeps up. Shindo’s vertical concept attacks the problem from another angle. Stacking GPU over HBM gives the shortest wiring, but it puts two heat sources on top of each other, so both faces need liquid cooling, and the glass-ceramic interposer’s tunable thermal expansion is meant to tame warpage. For now it is a concept with samples, from a company with no prior semiconductor substrate business. My read: through 2026 and 2027, the winner is whoever can yield 12-high and 16-high stacks on existing bonding. Hybrid bonding and true 3D GPU-memory stacking belong to HBM4E, HBM5 and beyond.

Open questions

  • SK hynix has not confirmed volume shipments of 16-high HBM4 for Rubin.
  • Samsung’s “more than 20%” for HCB and SK hynix’s “about 30%” for iHBM are vendor claims with undisclosed test conditions, so they cannot be compared directly.
  • The 825–900 µm height target appears only in industry analysis, with no published JEDEC document yet.
  • Shindo has not released thermal simulations or yield data for its 3D stack.
  • Per-pin rates, total thickness, interposer area and material conductivities are my arithmetic or textbook orders of magnitude, not vendor data.

Related videos

Your call: will the next HBM breakthrough come from hybrid bonding, better thermal materials, or stacking the GPU and memory vertically? Tell me why in the comments.

← Newer articles Older articles →