AI Inference Hardware · Page 1

NVIDIA Hopper, Blackwell, and the Rubin Roadmap: H100/H200/B200/GB200 NVL72 to Vera Rubin NVL72

H100 and H200 are the Hopper-generation parts NVIDIA ships in volume today; B200 and the rack-scale GB200 NVL72 are Blackwell, the dual-die, liquid-cooled successor that treats a 72-GPU rack as the unit of sale; Vera Rubin NVL72 is the platform NVIDIA has already specified, priced in performance claims, and dated for the second half of 2026. This module works through the fourth- and second-generation Transformer Engines, the FP8/NVFP4 numerics underneath the marketing multipliers, and the NVLink topology that turns 72 discrete GPUs into one memory domain — then decomposes NVIDIA's own "30x" GB200-vs-H100 inference claim into the arithmetic that actually produces it. After reading, you can choose between H100, H200, B200, and GB200 NVL72 from architecture alone, reconstruct any NVIDIA-published throughput multiplier from its per-GPU and per-rack components, and state precisely which Rubin-era numbers are shipped-and-measured versus vendor-projected.

Deep dive 35 min read 17 sources As of 2026-09-17
Three NVIDIA data-center GPU generations on one timeline: Hopper (2022), Blackwell (2024), Rubin (second half of 2026, not yet shipped), with per-GPU NVLink bandwidth, HBM bandwidth, and low-precision tensor throughput at each step Hopper H100 SXM5 NVLink4: 900 GB/s HBM3: 80 GB @ 3.35 TB/s FP8 (sparse): 3.96 PFLOPS Transistors: 80B, TSMC 4N single monolithic die 4th-gen Tensor Core 2022 Blackwell GB200 NVL72, per GPU NVLink5: 1.8 TB/s HBM3e: 186 GB @ 8 TB/s FP4 (sparse): 20 PFLOPS Transistors: 208B, TSMC 4NP dual reticle die + NV-HBI 2nd-gen Transformer Engine 2024 Rubin Vera Rubin NVL72, per GPU NVLink6: 3.6 TB/s HBM4: 288 GB @ 22 TB/s NVFP4: 50 PFLOPS Transistors: 336B, dual-die 3rd-gen Transformer Engine vendor-projected, not yet shipped 2H 2026
Figure 1. Every generation doubles per-GPU NVLink bandwidth exactly (900 GB/s → 1.8 TB/s → 3.6 TB/s) while the numeric format used for the headline "low-precision" figure changes each time (FP8 → FP4 → NVFP4), which is itself part of why cross-generation throughput multipliers are hard to compare directly (section 4). Hopper figures from NVIDIA's H100 product page1; Blackwell figures are the GB200 NVL72 rack totals divided by 72 GPUs2; Rubin figures from NVIDIA's Rubin newsroom release and Rubin GPU architecture blog34.

1. Hopper: the fourth-generation tensor core and the FP8 Transformer Engine

1.1 SM anatomy: what actually changed physically

The Hopper generation is one die, GH100: 80 billion transistors on a custom TSMC 4N process, 814 mm², organized as 8 GPU Processing Clusters (GPCs) and, at full implementation, 72 Texture Processing Clusters (TPCs) — 9 per GPC, 2 Streaming Multiprocessors (SMs) per TPC, 144 SMs total5. No shipping H100 uses the whole die. The SXM5 board activates 132 of the 144 physical SMs (8 GPCs, 66 TPCs, 16,896 FP32 CUDA cores, 528 fourth-generation Tensor Cores, 50 MB L2, 80 GB HBM3 across 5 stacks); the PCIe Gen5 card activates 114 SMs (456 Tensor Cores, 80 GB HBM2e instead of HBM3)5. The gap between 144 and 132/114 is ordinary yield binning — NVIDIA reserves the full-reticle part for internal use and ships the cut-down dies as product, the same pattern every vendor in this track uses at every process node.

Each SM is not a single execution unit but four identical partitions, and the fourth-generation Tensor Core lives inside each one. A partition carries its own warp scheduler (32 threads/clock), dispatch unit, a 16,384-entry 32-bit register file, a mix of INT32/FP32/FP64 ALU lanes, and exactly one Tensor Core5 — so an SXM5 SM has 4 Tensor Cores and an SXM5 GPU has 528. On a per-SM basis the new Tensor Core delivers 2x the matrix-multiply-accumulate (MMA) rate of an A100 SM at equivalent data types, and 4x the rate of A100's 16-bit floating-point options once you also switch to the new FP8 input type — two separate multipliers (architecture, then precision) that compound5. A structured-sparsity feature doubles standard Tensor Core throughput again when a network's weight matrix has the required fine-grained 2:4 zero pattern5 — this is the "sparse" figure vendors quote as the headline number and the "dense" figure is half of it; keep the distinction straight when reading any spec sheet in this module, because NVIDIA, AMD, and every other vendor in this track apply the identical 2x sparsity convention to their own headline TFLOPS.

1.2 FP8 numerics and the Transformer Engine's per-layer dynamic scaling

Hopper adds two FP8 formats to the Tensor Core's repertoire: E4M3 (4 exponent bits, 3 mantissa bits, 1 sign bit) and E5M2 (5 exponent, 2 mantissa, 1 sign)5. E4M3 trades dynamic range for precision; E5M2 trades precision for range — the same sign/exponent/mantissa allocation choice you already know from BF16 vs FP16, one exponent bit narrower on both sides. Both halve the storage of FP16/BF16 and, combined with the architectural gains above, the H100 Tensor Core reaches 6x the throughput of an A100 FP16 Tensor Core on the same silicon area5.

FP8's problem is range: an E4M3 value saturates far sooner than FP32 or even FP16, so naively casting a transformer layer's activations to FP8 clips outliers and degrades accuracy. The Transformer Engine is NVIDIA's answer — not a separate execution unit but software-plus-hardware built around the Tensor Core that manages precision automatically, layer by layer. Concretely, after each layer the engine's Range Analysis block reads the statistics of the Tensor Core's output values; from those statistics it computes a scaling factor and a target format (FP8 or FP16) for the tensor; the Adaptive Format Conversion block then casts the tensor into that format before it is written back and consumed by the next layer's multiply-accumulate5. NVIDIA's own description: "every layer operates with exactly the range it requires and is accelerated in an optimal manner"5. This per-layer, per-tensor dynamic rescaling is the mechanism the "30x faster AI inference" and "9x faster AI training" H100-vs-A100 figures trace back to5 — note this is a different "30x" than the GB200-vs-H100 figure this module spends section 4 decomposing; NVIDIA has now used the same round number for two unrelated generational comparisons, a pattern worth remembering going in.

Left: one SM partition with its warp scheduler, register file, ALU lanes, and fourth-generation Tensor Core. Right: the Transformer Engine's per-layer loop between the Tensor Core, Range Analysis, and Adaptive Format Conversion SM partition (1 of 4) Warp Scheduler (32 thread/clk) Dispatch Unit Register File — 16,384 × 32-bit INT32 / FP32 / FP64 ALU lanes 128 FP32 lanes per SM partition (512 FP32 lanes per full SM) 4th-generation Tensor Core 2x MMA/SM vs A100; 4x with FP8 in ×4 partitions/SM, ×132 SMs on H100 SXM5 Transformer Engine (per layer) RangeAnalysis Adaptive FormatConversion H100 Tensor Coremultiply → accumulate FP32/FP16 output statistics scale + format FP8/FP16 tensor, next layer repeats every Transformer layer, independently per tensor
Figure 2. Left: one of Hopper's four identical SM partitions — warp scheduler, register file, ALU lanes, and one fourth-generation Tensor Core; a full H100 SXM5 SM has four of these, and the GPU has 132 SMs. Right: the Transformer Engine's per-layer loop — it reads the Tensor Core's output statistics, computes a scaling factor and target format for the next layer, and casts before the next multiply-accumulate. Source: NVIDIA H100 architecture whitepaper, Figures 6, 7, and 255.

1.3 HBM3 vs HBM3e: H100 vs H200

H200 is not a new die. It reuses the GH100 compute with the same 132-SM SXM5 configuration and the same FP8 Tensor Core throughput — NVIDIA's own H200 sheet lists an identical 3,958 TFLOPS FP8 (with sparsity)6 to the H100 SXM figure1. What changes is the memory stack: H100 SXM ships 80 GB of HBM3 at 3.35 TB/s1; H200 ships 141 GB of HBM3e at 4.8 TB/s6 — "nearly double the capacity of the NVIDIA H100 GPU with 1.4X more memory bandwidth"6. It is a memory-only refresh, the same category of upgrade AMD later applies to MI325X over MI300X (page 2).

Because H100 and H200 share compute, the entire H200 gain is a memory-bandwidth story, and it lands exactly where you'd expect: NVIDIA states H200 is "up to 1.9X Faster" on Llama 2 70B inference and "1.6X Faster" on GPT-3 175B inference versus H1006. Prefill is compute-bound and gets essentially nothing from the memory upgrade; decode is bandwidth-bound on reading the KV cache and model weights back from HBM every step, so a 1.4x bandwidth increase shows up almost directly as decode-throughput headroom — more so once the extra 61 GB of capacity is spent on a larger batch or a longer context rather than left idle. H200 NVL, the dual-slot air-cooled variant sold in pairs, claims "1.7x" LLM inference speedup and "1.3x more performance" on HPC workloads over H100 NVL6, the same asymmetry: memory-bound inference gains more than compute-bound HPC.

1.4 Form factors: SXM vs NVL vs PCIe

Three H100 SKUs trade SM count, memory, and interconnect against thermal and mechanical constraints:

  • H100 SXM5: 132 SMs, 80 GB HBM3 @ 3.35 TB/s, 900 GB/s NVLink4, up to 700 W configurable TDP1 — the reference part for 8-GPU HGX baseboards; an 8-way HGX H100 reaches "over 32 petaFLOPS of deep learning compute performance using sparse FP8 operations"5 (8 × 3,958 TFLOPS ≈ 31.7 PFLOPS, matching NVIDIA's rounding).
  • H100 NVL: a dual-GPU board pairing two cut-down dies with an NVLink bridge, 94 GB HBM3 @ 3.9 TB/s per GPU, 600 GB/s NVLink, 350–400 W configurable TDP1 — positioned for inference deployments that want more memory per rack-unit and don't need the full SXM baseboard's scale-up domain.
  • H100 PCIe: 114 SMs, 456 Tensor Cores, 80 GB HBM2e (not HBM3) across 5 stacks5, 128 GB/s PCIe Gen5 x16 host link5 — the standard double-width card for servers that were not built around SXM's proprietary socket and higher power delivery; it gives up 18 SMs, HBM3's extra bandwidth, and most of NVLink's bandwidth relative to SXM5.

1.5 NVLink4 and NVSwitch3: the Hopper scale-up ceiling

Fourth-generation NVLink delivers 1.5x the per-link bandwidth of third-generation NVLink (A100): 18 links per H100 GPU, each carrying 25 GB/s in each direction over two high-speed differential pairs, for 900 GB/s total bidirectional bandwidth — 7x a PCIe Gen5 x16 link's 128 GB/s5. Third-generation NVSwitch, the chip that connects those links across GPUs, exposes 64 ports of fourth-generation NVLink per switch for 13.6 Tbit/s of total switch throughput (up from 7.2 Tbit/s the prior generation), and adds in-network SHARP reductions (write-broadcast, reduce-scatter, and broadcast atomics) that cut collective-communication latency versus running the same all-gather/reduce-scatter through software on A1005.

Stacking NVSwitches into a second network tier — the NVLink Switch System — extends this beyond one 8-GPU node to up to 256 GPUs sharing one address space, delivering 57.6 TB/s of all-to-all bandwidth and up to 1 exaFLOP of sparse FP8 compute across the domain5. NVIDIA's own 32-node, 256-GPU DGX SuperPOD comparison quantifies what that buys relative to an A100 SuperPOD wired over InfiniBand: dense PFLOP/s rises from 80 to 512 (6.4x, purely the H100-vs-A100 compute delta), bisection bandwidth rises from 6,400 to 57,600 GB/s (9x), and all-reduce bandwidth rises from 100 to 450 GB/s (4.5x)5 — the bisection and reduce multipliers are the interconnect's contribution alone, independent of the compute upgrade, and they are the direct antecedent of the "9x more GPUs, higher bisection bandwidth" argument Blackwell's NVL72 domain (section 3) pushes one level further by putting the switches inside one rack instead of across a data-center network.

2. Blackwell: the dual-die package and NVFP4

2.1 The reticle limit and NV-HBI

Where Hopper is one die, Blackwell is two. Each Blackwell compute die is fabricated at essentially the largest area a single lithography exposure (a "reticle") can pattern — NVIDIA's own description calls them "two reticle-sized dies"7 — joined by NV-HBI (NVIDIA High-Bandwidth Interface), "a custom, power-efficient die-to-die interconnect technology that provides 10 TB/s of bandwidth"7, matching the "10 terabytes per second (TB/s) chip-to-chip interconnect" figure on NVIDIA's Blackwell architecture page8. The two dies present to software as a single GPU: 208 billion transistors total, manufactured on TSMC's 4NP process87, scheduled and addressed as one CUDA device rather than two GPUs bridged by an external interconnect. The engineering logic is the reticle limit itself: once a design needs more transistors than one exposure can produce, the choice is to shrink the design or split it across two dies and make the join fast enough that it disappears from the programming model's point of view. NVIDIA took the second path, at 10 TB/s — roughly 11x a single H100's total NVLink4 bandwidth, spent entirely on keeping two dies coherent rather than reaching other GPUs.

For a concrete sense of how the physical package maps to compute units: NVIDIA's Blackwell Ultra variant (the GB300-generation part, which shares the same die-pair design) packs 160 SMs across the two dies in 8 GPCs, with 4 fifth-generation Tensor Cores per SM — 640 Tensor Cores total — delivering 15 PFLOPS of dense NVFP47. Blackwell Ultra itself runs "2X the attention-layer acceleration and 1.5X more AI compute FLOPS" versus the original Blackwell die pair8, so read the 160-SM/640-Tensor-Core/15-PFLOPS figures as Blackwell Ultra's specific configuration, not the baseline B200/GB200 numbers this module otherwise quotes from the GB200 NVL72 product page (Table 1, section 3).

Two reticle-sized Blackwell dies joined by the NV-HBI interconnect at 10 TB/s, each with 8 GPCs and flanking HBM3e stacks, presenting to software as one GPU One Blackwell GPU package Die A 8 GPCs per die (Blackwell Ultra: 160 SMs total) Die B 8 GPCs per die (Blackwell Ultra: 160 SMs total) NV-HBI 10 TB/s HBM3e stacks — up to 192 GB @ 8 TB/s (B200) TSMC 4NP · 208B transistors total, ~104B per die
Figure 3. Blackwell's two reticle-sized dies act as one GPU because NV-HBI's 10 TB/s die-to-die link is fast enough to hide the seam from CUDA. GPC/SM counts shown are Blackwell Ultra's published configuration7; die-to-die bandwidth and transistor count from NVIDIA's Blackwell architecture page8; HBM3e capacity/bandwidth is the DGX B200 system total divided by 8 GPUs9.

2.2 NVFP4 and the second-generation Transformer Engine

NVFP4 is a 4-bit floating-point format, E2M1: 1 sign bit, 2 exponent bits, 1 mantissa bit, with a maximum representable magnitude of ±610. On its own, 4 bits of mantissa-plus-exponent is far too coarse to represent a transformer's activations directly — the fix, as with Hopper's FP8, is scaling, but Blackwell's second-generation Transformer Engine pushes the scaling granularity down from per-tensor to per-block. NVFP4's default recipe groups 16 consecutive elements into one micro-scaling block, stores an FP8 E4M3 scale factor per block, and layers a single FP32 scale factor on top of the entire tensor: the dequantized value is x = x_e2m1 × s_block × s_global10. By default weights use 2D block scaling (16×16 tiles) and activations/gradients use 1D scaling (16 consecutive elements along a row), with stochastic rather than round-to-nearest-even rounding applied when casting into the format10. A 16-element block scale is finer-grained than the 32-element blocks and single shared E8M0 exponent that the competing MXFP4 format (used by AMD's CDNA4, page 2) uses; tighter blocks isolate outliers better, at the cost of one FP8 scale factor per 16 values instead of per 32.

NVIDIA frames native FP4 as doubling both performance and the model size that fits in memory relative to running the same weights at FP88 — each FP4 value is half the storage of an FP8 value, and the Tensor Core does correspondingly more MACs per cycle at 4-bit than at 8-bit, the identical multiplicative logic Hopper's FP8 Tensor Core exploited over FP16 (section 1.1). Blackwell Ultra's incremental FP4-specific gain on top of that — "2X the attention-layer acceleration"8 — comes from further Tensor Core throughput increases at the silicon level rather than from the number format itself, since Ultra uses the same NVFP4 layout as standard Blackwell.

2.3 NVLink5 and NVSwitch: the 130 TB/s NVL72 domain

Fifth-generation NVLink doubles Hopper's per-GPU bandwidth again, to 1.8 TB/s bidirectional per GPU8 — matched exactly by the DGX B200 system's stated 14.4 TB/s aggregate NVLink bandwidth divided across 8 GPUs9. The fourth-generation NVSwitch chip inside a Blackwell rack adds hardware SHARP reductions specifically tuned for FP8, claimed at "4X bandwidth efficiency"8 over running the same collective in software. Wired into a single 72-GPU NVL72 domain, that per-GPU bandwidth aggregates to 130 TB/s of GPU-to-GPU bandwidth inside one rack8, and the same NVLink Switch technology can bridge multiple racks to up to 576 GPUs at a reduced 1.8 TB/s per-GPU rate across servers8 — the NVL72 domain is the tier where full per-GPU bandwidth holds; anything larger drops to inter-rack rates. Section 3 works through what a full NVL72 rack looks like when you add the CPUs and the memory back in.

That 130 TB/s domain is a different kind of fabric from the InfiniBand or Ethernet that connects racks to each other, not just a faster version of the same thing. NVLink is memory-semantic: every GPU in the domain can read and write every other GPU's memory directly, which is what lets the rack behave as one addressable pool (section 3.2). InfiniBand and Ethernet are message-semantic, moving data through send/receive operations a software collective library has to orchestrate explicitly. The bandwidth gap reflects that difference in kind as much as in degree: a single NDR InfiniBand link runs at 400 Gb/s (50 GB/s)5, roughly two orders of magnitude below one GPU's 1.8 TB/s NVLink5 allocation. That is one reason the GB200 NVL72 "30x" comparison (section 4) networks its H100 baseline over InfiniBand specifically: an InfiniBand-scaled H100 cluster is real, purchasable hardware, but comparing it to an NVL72 rack means comparing two different interconnect semantics as much as two different generations of GPU.

Table 1. Per-GPU spec comparison across shipping NVIDIA data-center parts. H100/H200 rows from NVIDIA's own product pages16; B200 and GB200 rows are per-GPU shares of published 8-GPU (DGX B200) and 72-GPU (GB200 NVL72) system totals, divided down and flagged where derived9211.
PartMemory (GB)Mem BW (TB/s)NVLink (TB/s/GPU)FP8 dense (PFLOPS)FP8 sparse (PFLOPS)NVFP4 dense (PFLOPS)NVFP4 sparse (PFLOPS)TDP (W)
H100 SXM5803.350.91.979*3.958n/an/a≤700
H200 SXM1414.80.91.979*3.958n/an/a≤700
B200 (DGX B200 ÷ 8)1808.01.84.5†9918n/a§
GB200 (NVL72 ÷ 72)1868.01.85†101020n/a§

* Derived: NVIDIA's product page lists only the sparse figure for H100/H200; dense is estimated at sparse ÷ 2 per the general Tensor Core sparsity mechanism5, not a directly published H100/H200 dense number. † Derived the same way for B200/GB200 FP8, but confirmed explicitly for NVFP4 and for B200's FP8 by NVIDIA's own DGX B200 footnote: "Specification shown in sparse. Dense performance is ½ sparse spec shown"9. § Neither the DGX B200 nor GB200 NVL72 page states a per-GPU TDP; DGX B200's system total is "~14.3 kW max" for 8 GPUs plus 2 host CPUs9, not divisible into a clean per-GPU figure.

3. GB200 NVL72: the rack is the product

3.1 Rack anatomy: 36 Grace CPUs, 72 Blackwell GPUs

A GB200 NVL72 rack is built from 36 identical GB200 Grace Blackwell Superchips. Each Superchip pairs one Grace CPU with two Blackwell GPUs over NVLink-C2C at up to 3.6 TB/s, and carries 372 GB of HBM3e at 16 TB/s addressable across its two GPUs2. Multiply by 36 and NVIDIA's own configuration line reads exactly "36 Grace CPU | 72 Blackwell GPUs"2 — the Superchip, not the individual GPU, is the rack's building block, and every GPU-count multiplier this module discusses (section 4) is ultimately a multiplier on 36 of these units, not on 72 independent cards. Buying below rack scale means buying Superchips, not individual Blackwell GPUs, and the Grace CPU's LPDDR5X capacity comes along with every purchase whether or not the workload needs CPU-side memory — a fixed-ratio bundling H100/H200 (sold as GPU-only SXM/PCIe/NVL boards, section 1.4) does not impose.

36 GB200 Superchips, each one Grace CPU plus two Blackwell GPUs, wired into a single 130 TB/s NVSwitch fabric that forms one NVL72 rack GB200 NVL72 — one rack, one NVLink domain 1 Grace+2 Blackwell 1 Grace+2 Blackwell 1 Grace+2 Blackwell 1 Grace+2 Blackwell 1 Grace+2 Blackwell 1 Grace+2 Blackwell × 36 Superchips total = 36 Grace CPUs + 72 Blackwell GPUs NVSwitch fabric — 130 TB/s NVLink5 domain every GPU addresses every other GPU's memory 13.4 TB HBM3e@ 576 TB/s (72 GPUs) 17 TB LPDDR5X@ 14 TB/s (36 CPUs) 1,440 PFLOPSNVFP4, sparse ≈30 TBunified memory (HBM+CPU) 2,592 Arm Neoverse V2 CPU cores total (72 per Grace CPU)
Figure 4. A GB200 NVL72 rack is 36 Superchips (1 Grace CPU + 2 Blackwell GPUs each) sharing one 130 TB/s NVLink5 domain. Superchip and rack totals from NVIDIA's GB200 NVL72 product page2; the ≈30 TB unified-memory figure is HBM3e plus LPDDR5X added together (section 3.2), matching NVIDIA's separately stated 30 TB claim11.

3.2 Aggregate arithmetic: the rack totals reconcile

Sum the 36 Superchips' components and NVIDIA's published rack totals fall out directly, which is a useful check on any rack-scale spec sheet. GPU side: 72 Blackwell GPUs carry 13.4 TB of HBM3e at an aggregate 576 TB/s2 — 186 GB and 8 TB/s per GPU (Table 1), and 72 × 186 GB ≈ 13.4 TB, 72 × 8 TB/s = 576 TB/s, exactly what the rack figure states. CPU side: 36 Grace CPUs (2,592 Arm Neoverse V2 cores total, 72 per CPU) carry 17 TB of LPDDR5X at 14 TB/s aggregate2. Add the two memory pools and you land almost exactly on a separate NVIDIA figure: 13.4 + 17 = 30.4 TB, matching "30 TB of unified memory" per rack stated on NVIDIA's GB200 NVL72 developer blog11 — CPU and GPU memory are coherently addressable across NVLink-C2C and the NVLink5 fabric together, so the rack behaves as one 30 TB memory domain rather than two pools a programmer manages separately.

Compute totals follow the identical per-GPU × 72 arithmetic: 1,440/720 PFLOPS NVFP4 sparse/dense, 720/360 PFLOPS FP8 sparse/dense, 360/180 PFLOPS FP16/BF16 sparse/dense, all inside the rack2, and all reachable from any of the 72 GPUs across the 130 TB/s NVLink5 domain (section 2.3) without a single collective operation leaving the rack. That property — one memory domain, one NVLink fabric, 72 GPUs behind it — is exactly what section 4 has to account for when it decomposes NVIDIA's headline throughput multiplier: some of the gain is Blackwell's architecture, and some of it is simply that the comparison baseline, an 8-GPU HGX H100 node, has no equivalent to this fabric at all.

4. Decomposing NVIDIA's "30x inference speedup" claim

4.1 The headline and its footnote

NVIDIA's GB200 NVL72 product page states the rack delivers "30x vs. NVIDIA H100 GPU (LLM Inference)"2, attached to a footnote specifying the exact benchmark configuration behind it: "TTL = 50 milliseconds (ms) real time, FTL = 5s, 32,768 input/1,024 output, NVIDIA HGX H100 scaled over InfiniBand (IB) vs. GB200 NVL72"2 — a 50 ms time-per-output-token target, a 5-second time-to-first-token budget, a 32,768-token input prompt, and 1,024 output tokens, comparing an InfiniBand-networked cluster of H100 HGX nodes against one GB200 NVL72 rack. The same "up to 30x" number, attached to a different underlying benchmark (GPT-MoE-1.8T rather than the token-length/latency profile above) and a different pair of per-GPU throughput figures, appears separately on NVIDIA's GB200 NVL72 developer blog11. Neither source states plainly whether the H100 comparator is scaled to 72 GPUs to match the rack's GPU count or held at some smaller node size with the multiplier already reflecting node-count scaling — that ambiguity is the first thing to resolve before treating "30x" as one well-defined ratio.

The token-length and latency numbers in that footnote are not incidental. A 32,768-token input prompt is a substantial prefill cost on its own, and a 5-second time-to-first-token budget caps how much prefill latency a system can spend before the first output token has to appear; at any non-trivial concurrency, satisfying that budget forces a smaller per-GPU batch size than the hardware's raw FLOPS would otherwise support, on whichever side of the comparison has less memory to hold concurrent requests' KV caches. GB200 NVL72's advantage under this specific configuration is therefore not only that a Blackwell GPU computes faster than an H100 GPU; it is that 186 GB of HBM3e per GPU (Table 1) versus H100's 80 GB changes what batch size is achievable under the same latency ceiling before any difference in raw Tensor Core throughput is even measured. That is a third factor baked into "30x," alongside architecture and GPU count, and it is specific to this token-length/latency configuration — a shorter prompt or a looser TTFT budget would narrow it.

4.2 The arithmetic that reconciles — for one benchmark

NVIDIA's MLPerf Inference v5.0 submission gives the cleanest accounting available anywhere in this module's source set. On the Llama 3.1 405B benchmark introduced that round, NVIDIA reports "GB200 NVL72 delivers up to 3.4x higher per-GPU performance...compared to the H200 eight-GPU system," and explains the headline directly: "GB200 NVL72 increases performance by up to 30x, through a combination of much higher per-GPU performance as well as 9x more GPUs in the system"12. The arithmetic is explicit and it reconciles almost exactly: 3.4x (the audited, MLPerf-submitted per-GPU gain) × 9x (72-GPU rack ÷ 8-GPU HGX baseline) = 30.6x ≈ "up to 30x." This is the one place in NVIDIA's public materials where the full multiplier is attributable to two named, independently verifiable factors — a real architecture gain and a real rack-size gain, multiplied together rather than either one alone.

Waterfall showing NVIDIA's own MLPerf-submitted numbers decomposing up to 30x into a 3.4x per-GPU architecture gain multiplied by a 9x GPU-count effect Decomposing "up to 30x" — Llama 3.1 405B, MLPerf Inference v5.0 1x H200, 8-GPU system (MLPerf baseline) 3.4x same per-GPU rate, 72 GPUs, architecture gain only 30.6x GB200 NVL72, 72 GPUs ≈ NVIDIA's "up to 30x" ×3.4 per-GPU architecture gain ×9 GPU count (72 ÷ 8)
Figure 5. NVIDIA's own MLPerf Inference v5.0 blog states both factors directly: a 3.4x per-GPU gain on Llama 3.1 405B and a 9x GPU-count effect from comparing a 72-GPU rack to an 8-GPU baseline system; multiplied together they land almost exactly on the "up to 30x" headline12. This decomposition holds for this one benchmark only — section 4.3 shows it does not carry over to NVIDIA's other "30x" claims.

4.3 Where the arithmetic doesn't reconcile

The clean 3.4x × 9x ≈ 30.6x decomposition is specific to one benchmark: Llama 3.1 405B under MLPerf's server scenario. It does not generalize to NVIDIA's other "30x" claims. On the GPT-MoE-1.8T workload cited on the GB200 NVL72 developer blog, NVIDIA reports "GB200 at 150 tokens/sec/GPU and H100 at 3.4 tokens/sec/GPU"11 — a 44.1x per-GPU ratio — right alongside the identical "30x speedup" headline11. If the same 9x-GPU-count decomposition applied here, 44.1 × 9 would be roughly 397x, not 30x; if the comparison is instead already GPU-for-GPU at matched rack scale with no separate count multiplier, 44.1x itself still does not equal 30x. NVIDIA does not publish the intermediate configuration — batch size, sequence length, or the H100 side's network topology — needed to reconcile the two numbers on that page, so the GPT-MoE-1.8T "30x" and the Llama 3.1 405B "30x" are two different, non-comparable measurements that happen to round to the same headline figure.

The pattern predates Blackwell. NVIDIA's own H100 architecture whitepaper states the Transformer Engine delivers "up to 9x faster AI training and 30x faster AI inference speedups on large language models compared to the prior generation A100"5 — a claim about a completely different chip pair (H100 vs A100), a completely different mechanism (per-layer FP8 dynamic scaling, not a GPU-count effect at all, since the comparison in that whitepaper's own chart is against an equal-scale InfiniBand cluster), and a completely different workload (Megatron-Turing NLG 530B under a latency ceiling)5. Across three NVIDIA documents spanning two full GPU generations, "30x" recurs as the round number attached to whichever inference comparison currently produces something close to it, not as a fixed multiplier with one derivation you can carry from document to document.

4.4 An independent decomposition

An independent analysis of the original Blackwell inference claims, published shortly after Blackwell's GTC 2024 unveiling, reaches a comparable conclusion by a different route. Once quantization is held constant across the comparison — NVIDIA's headline compares GB200 at FP4 against H200 at FP8, even though Blackwell also runs FP8 — the analysis finds "we have only an ~18x performance gain from the H200 to the GB200"13, roughly 40% below the "30x" headline. It also examines the specific latency and token-count configuration in NVIDIA's own footnote (the same 32,768-input/1,024-output, 5-second-TTFT numbers quoted in section 4.1) and argues it was chosen to disadvantage H200's available parallelism strategies specifically: under a different, equally realistic configuration (512 input tokens, 2,048 output tokens) the same analysis finds "performance gains are less than 8x," concluding NVIDIA selected "the worst possible parallelism scheme to run on the H200" for the baseline13.

None of this disputes that GB200 NVL72 is faster than an equivalent H200 deployment — it is, on every source in this module. It disputes only that any single number captures it: the multiplier depends on which precision, which token-length profile, and which H200 parallelism strategy you hold fixed, and NVIDIA's footnoted configuration is one specific point in that space rather than a property of the silicon. Section 6 (Rubin) carries the same caveat forward for every "10x" and "4x" figure NVIDIA has published for a platform that has not shipped yet.

5. The precision ladder: FP16/BF16 → FP8 → NVFP4

Every step down this ladder halves the storage per value and, on the Tensor Core generation that supports it, doubles peak throughput — an architectural multiplier (more multiply-accumulates per cycle at a narrower bit width) rather than something specific to any one format. On a single GB200 GPU the sparse peak doubles exactly at each step: 5 PFLOPS FP16/BF16 → 10 PFLOPS FP8 → 20 PFLOPS NVFP4 (Table 1)2. What changes at each step — and what makes each step usable without destroying model accuracy — is scaling granularity: how finely the numeric range gets recalibrated to the data actually being represented.

FP16 (1 sign, 5 exponent, 10 mantissa bits) and BF16 (1 sign, 8 exponent, 7 mantissa bits) need no dynamic scaling; both cover a wide enough range natively that a transformer's activations and gradients fit without rescaling. FP8 removes that safety margin: two encodings split the remaining 8 bits differently, "E5M2 uses five bits for the exponent and two bits for the mantissa and is a truncated IEEE FP16 format," while "in circumstances where more precision is required at the expense of some numerical range, the E4M3 format makes a few adjustments to extend the range representable with a four-bit exponent and a three-bit mantissa"5 — E4M3 additionally forgoes representing infinities and reserves only one bit pattern for NaN specifically to claw back extra range for finite values14. That narrower native range is why Hopper's Transformer Engine (section 1.2) recalibrates a scale factor once per tensor, per layer, rather than relying on FP8's fixed exponent range alone.

NVFP4 goes further on both axes at once: E2M1 has only 1 sign, 2 exponent, and 1 mantissa bit, with a maximum representable magnitude of ±610 — not enough dynamic range for a single scale factor to cover an entire tensor without either clipping outliers or wasting most of the format's 16 representable magnitudes on values that never occur. The fix is finer-grained scaling: a local FP8 E4M3 scale factor covers just 16 consecutive elements, and one FP32 scale factor covers the tensor as a whole on top of that, so the dequantized value is x = x_e2m1 × s_block × s_global10. The scaling granularity gets finer exactly in proportion to how little native range each format has left: none for FP16/BF16, once per tensor per layer for FP8, once per 16 values (plus once per tensor) for NVFP4.

Precision ladder from FP16/BF16 through FP8 to NVFP4, showing bit width, scaling granularity, and peak sparse throughput per GPU on GB200 FP16 / BF16 16 bits e5m10 or e8m7 scaling: none (native range) 5 PFLOPS GB200, per GPU, sparse ×2 FP8 8 bits E4M3 or E5M2 scaling: per tensor, per layer (dynamic) 10 PFLOPS GB200, per GPU, sparse ×2 NVFP4 4 bits E2M1, max ±6 scaling: 16-value block (FP8 scale) + tensor (FP32) 20 PFLOPS GB200, per GPU, sparse Scaling granularity gets finer at exactly the rate native dynamic range shrinks
Figure 6. Bit width halves and peak sparse throughput doubles at each step (GB200, per GPU, from Table 1)2; the scaling mechanism gets correspondingly finer-grained — none, per-tensor, per-16-value-block — to compensate for the shrinking native dynamic range510.

6. The Rubin / Vera Rubin roadmap

6.1 Platform components

NVIDIA announced the Rubin platform on 2026-01-05 as the next full generational refresh after Blackwell: a Vera CPU, a Rubin GPU, sixth-generation NVLink switch silicon, a ConnectX-9 SuperNIC, a BlueField-4 DPU, and a Spectrum-6 Ethernet switch, sold as rack-scale systems rather than individual parts — the same organizing principle Blackwell introduced with GB200 NVL72 (section 3)3. The announcement also folds in Groq's LPU technology following NVIDIA's acquisition of Groq, described as "the newly integrated NVIDIA Groq 3 LPU"3, though the newsroom release gives no technical detail on how Groq silicon fits into the Vera/Rubin stack beyond naming it.

The Rubin GPU itself extends Blackwell's dual-die construction (section 2.1): two dies joined by NV-HBI, 336 billion transistors total, 224 SMs, and 896 third-generation-Transformer-Engine Tensor Cores4 — up from Blackwell's 208 billion transistors and (on the Blackwell Ultra configuration) 160 SMs/640 Tensor Cores. It carries up to 288 GB of HBM4 at up to 22 TB/s, "a 2.8x increase over Blackwell and Blackwell Ultra"4, and delivers 50 petaflops of NVFP4 compute for AI inference34, with NVIDIA additionally claiming 2x FP32 and 4x BF16/FP16 throughput and doubled K-dimension instruction throughput relative to Blackwell's baseline4. Sixth-generation NVLink gives each Rubin GPU 3.6 TB/s of bandwidth — exactly double NVLink5's 1.8 TB/s per GPU (section 2.3), continuing the doubling-per-generation pattern from Figure 1 — and a Vera Rubin NVL72 rack aggregates that to 260 TB/s315, roughly double GB200 NVL72's 130 TB/s domain. Vera, the CPU half of the platform, carries 88 NVIDIA custom Olympus cores3 — up from Grace's 72 Arm Neoverse V2 cores per CPU (section 3.1) — and a full Vera CPU rack (256 CPUs) is stated to support "more than 22,500 concurrent sandbox environments"1615, the framing NVIDIA uses for reinforcement-learning and agent workloads rather than classic training or inference throughput.

The rest of the platform's naming reflects a similar consolidation instinct to Blackwell's dual-die package: BlueField-4, the DPU, "combines the Vera CPU and ConnectX-9 SuperNIC" into one processor15 rather than keeping host offload and networking as two separate chips, and each Rubin compute tray carries eight ConnectX-9 SuperNICs15 feeding a Spectrum-6 SPX Ethernet switch rated at 102.4 Tb/s across 512 lanes of 200 Gb/s each15 for scale-out beyond the NVLink6 domain. Below the 72-GPU Vera Rubin NVL72 rack, NVIDIA also lists HGX Rubin NVL8, an eight-Rubin-GPU server board without the Vera CPU pairing3 — the same smaller-scale, GPU-only building-block role HGX H100 (section 1.5) and DGX B200 (Table 1) play for their respective generations, for buyers who want Rubin silicon without committing to full rack-scale NVLink6 deployment.

Vera Rubin platform components, and Vera Rubin NVL72's rack-scale numbers roughly doubling GB200 NVL72's on NVLink domain bandwidth, per-GPU NVLink, and per-GPU NVFP4 compute Vera Rubin platform Vera CPU88 Olympus cores Rubin GPU336B transistors NVLink6Switch ConnectX-9SuperNIC BlueField-4DPU Spectrum-6Switch Vera Rubin NVL72 vs. GB200 NVL72 (rack-scale) GB200 NVL72 Vera Rubin NVL72 130 260 NVLink domain(TB/s / rack) 1.8 3.6 NVLink(TB/s / GPU) 20 50 NVFP4(PFLOPS / GPU)
Figure 7. Vera Rubin's platform components (top) and its rack-scale figures against GB200 NVL72 (bottom): NVLink domain bandwidth and per-GPU NVLink both exactly double; NVFP4 per GPU rises 2.5x, though NVIDIA does not state whether the 20 and 50 PFLOPS figures share the same sparse/dense basis2315. All Vera Rubin figures are vendor-projected ahead of shipment (section 6.4).

6.2 Rubin CPX: disaggregated prefill

Rubin CPX, announced 2025-09-09 (before the full Rubin platform announcement), is a separate GPU SKU built specifically for the prefill phase of long-context inference. Instead of HBM it carries 128 GB of GDDR7, and delivers "up to 30 petaflops of compute with NVFP4 precision"17 — less raw NVFP4 throughput than the full Rubin GPU's 50 PFLOPS, but at a fraction of the memory cost per gigabyte, because GDDR7 is commodity graphics memory rather than a stacked HBM module. NVIDIA claims Rubin CPX runs attention up to 3x faster than a GB300 NVL72 system17, and packages it into a dedicated rack, Vera Rubin NVL144 CPX, rated at 8 exaflops of AI performance, 100 TB of fast memory, and 1.7 petabytes/sec of memory bandwidth — "7.5x more AI performance than NVIDIA GB300 NVL72 systems"17.

The economic logic is a direct application of the prefill/decode split you already work with: prefill is compute-bound (attention and MLP FLOPs scale with context length, and a long prompt has to be processed once before any tokens generate), while decode is bandwidth-bound on reading weights and the KV cache back from memory every step. A GPU built for prefill does not need HBM's bandwidth-per-dollar; it needs FLOPs and enough capacity to hold activations for very long contexts, which is exactly what GDDR7 is cheaper at delivering than HBM4. Splitting the two phases across different GPU designs lets each one be over-provisioned for the resource it actually consumes instead of paying for HBM bandwidth on every GPU in the fleet, including the prefill GPUs that would leave most of it idle. NVIDIA's own framing of the payoff is explicitly economic: "$5B in token revenue for every $100M invested" in Rubin CPX infrastructure17 — a claim with no disclosed workload mix, utilization assumption, or pricing basis behind it, so treat the ratio itself, rather than the specific $5B/$100M framing, as the takeaway: disaggregating prefill from decode is worth doing because the two phases have genuinely different hardware requirements, not because of this particular number.

Disaggregated serving: a prompt is processed by the GDDR7-equipped Rubin CPX for prefill, then its KV cache hands off to an HBM4-equipped Rubin GPU for decode Prompt Rubin CPX prefill — compute-bound 128 GB GDDR7 (not HBM) 30 PFLOPS NVFP4 3x attention vs GB300 NVL72 KV cache Rubin GPU decode — bandwidth-bound 288 GB HBM4 @ 22 TB/s 50 PFLOPS NVFP4 Tokens GDDR7 is cheaper per GB where bandwidth matters less than FLOPs (prefill); HBM4 earns its cost where bandwidth is the bottleneck (decode)
Figure 8. Rubin CPX and the standard Rubin GPU split the two inference phases across memory technologies matched to what each phase is bound by. Specs from NVIDIA's Rubin CPX newsroom release and Rubin GPU architecture blog174.

6.3 What's shipped versus vendor-projected

NVIDIA's own language is not fully consistent about how far along Rubin is. The Vera Rubin platform page states the platform is "now in full production"16, while the January 2026 newsroom announcement gives an availability window of "second half of 2026" for the platform as a whole3 — read the first as a manufacturing-ramp claim and the second as when systems reach customers; neither source reconciles the two statements explicitly, and this module does not reconcile them either. Every performance figure attached to Rubin here — 50 PFLOPS NVFP4 per GPU, 260 TB/s per rack, 22 TB/s of HBM4 bandwidth, up to 10x lower inference token cost, 4x fewer GPUs to train MoE models, 8x inference speedup for Vera Rubin NVL4 versus Hopper, 7.5x more AI performance for the NVL144 CPX rack versus GB300 NVL72, 18x faster assembly/servicing from the second-generation RAS engine31617 — is a vendor projection made before independent measurement was possible, the same status Blackwell's figures had at GTC 2024, before section 4 could decompose them against real MLPerf submissions. Treat every Rubin-era multiplier the way section 4 treats "30x": as a specific, disclosed (or partially disclosed) comparison basis, not a settled property of hardware that has not yet shipped to independent benchmarkers.

Where experts disagree

Section 4 already showed that "up to 30x" reconciles cleanly for one benchmark (Llama 3.1 405B, MLPerf server scenario) and does not reconcile for at least two others NVIDIA has published under the same headline. The remaining disagreement is about how to read that pattern: is "30x" a legitimate, if narrow, systems-level metric that happens to get reused loosely, or is it primarily a benchmark-configuration artifact that should not anchor any capacity-planning arithmetic at all?

"30x" is a reproducible systems-level metric — for the benchmark it was measured on

NVIDIA's own MLPerf-grounded accounting122

For Llama 3.1 405B, the audited MLPerf-submitted 3.4x per-GPU gain multiplied by the documented 9x GPU-count ratio (72-GPU rack vs. 8-GPU baseline) lands at 30.6x, matching "up to 30x" almost exactly12. The per-GPU component is independently checkable against MLCommons' public results database, not a number NVIDIA invented for a press release, and the two-factor accounting is stated explicitly rather than left implicit.

  • Grounded in an audited, third-party-verified MLPerf submission for the per-GPU component.
  • Internally consistent and fully reconstructible from two named, disclosed factors, for this one workload.
  • Does not generalize: the identical headline reused for GPT-MoE-1.8T attaches to a per-GPU ratio (44x) that does not decompose the same way (section 4.3).
  • Conflates an architecture gain with a scale-out gain that any sufficiently larger GPU count would partly capture, NVLink domain or not.

"30x" is a benchmark-configuration artifact, not a chip property

Independent recomputation, SemiAnalysis13

Holding precision constant (FP4 vs. FP4, rather than NVIDIA's FP4-vs-FP8 comparison) and correcting the H200 parallelism strategy used as the baseline drops the real gain to roughly 18x; a different, equally realistic token-length profile than NVIDIA's footnoted 32,768/1,024 configuration drops it further, to under 8x13. NVIDIA's specific footnoted configuration, the analysis argues, was chosen because it disadvantages H200's available parallelism options rather than because it represents a typical deployment.

  • Controls for the precision mismatch (FP4 vs. FP8) that inflates the headline before any GPU-count effect is even applied.
  • Consistent with the broader pattern of "30x" recurring across three separate NVIDIA documents with three different, non-reconcilable derivations (section 4.3).
  • The recomputed 18x and sub-8x figures are themselves one alternative configuration each, not a neutral ground truth benchmark.
  • Published in April 2024, before GB200 NVL72 volume shipments and before MLPerf's later audited v5.0 submission existed to check the per-GPU claim independently (section 4.2).
Assessment

Use the audited MLPerf per-GPU number for the specific workload you care about (3.4x for Llama 3.1 405B server scenario) as your starting point, and multiply it by whatever GPU count you are actually comparing against — not necessarily 9x, which is an artifact of comparing a 72-GPU rack to an 8-GPU baseline system. Do not quote "30x" itself as a planning input; across three NVIDIA documents it attaches to at least three different, non-reconcilable measurements. What would settle the size of the real gain for any specific deployment is an apples-to-apples run at matched precision, matched token profile, and matched GPU count for both H200 and GB200 NVL72 — the kind of methodology InferenceMAX uses (page 5) — at the exact 32,768-input/1,024-output, 5-second-TTFT configuration NVIDIA's own footnote specifies; as of this research no such independent run at that specific configuration was available to check against.

References

  1. officialNVIDIA. H100 GPU (product page). 2026. https://www.nvidia.com/en-us/data-center/h100/. Accessed 2026-09-17. Used for: H100 SXM/NVL FP8/FP16/TF32 Tensor Core TFLOPS, HBM3 capacity/bandwidth, NVLink4 and PCIe Gen5 bandwidth, TDP.
  2. officialNVIDIA. GB200 NVL72 (product page). 2026. https://www.nvidia.com/en-us/data-center/gb200-nvl72/. Accessed 2026-09-17. Used for: GB200 NVL72 rack configuration (36 Grace CPU/72 Blackwell GPU), rack-level NVFP4/FP8/FP16 PFLOPS, 13.4TB HBM3e @ 576TB/s, 130TB/s NVLink, CPU core/memory totals, GB200 Superchip specs, the "30x/4x/25x" headline claims and TTL/FTL footnote.
  3. officialNVIDIA. NVIDIA Kicks Off the Next Generation of AI With Rubin — Six New Chips, One Incredible AI Supercomputer. NVIDIA Newsroom, 2026-01-05. https://nvidianews.nvidia.com/news/rubin-platform-ai-supercomputer. Accessed 2026-09-17. Used for: Rubin platform component list, NVLink6 3.6TB/s/GPU and 260TB/s/rack, Rubin GPU 50 PFLOPS NVFP4, Vera CPU 88 Olympus cores, Vera Rubin NVL72/HGX Rubin NVL8 configurations, "10x"/"4x" platform claims, RAS engine claim, 2H 2026 availability.
  4. blogNVIDIA. Inside NVIDIA Rubin GPU Architecture: Powering the Era of Agentic AI. NVIDIA Technical Blog, 2026-07-21. https://developer.nvidia.com/blog/inside-nvidia-rubin-gpu-architecture-powering-the-era-of-agentic-ai/. Accessed 2026-09-17. Used for: Rubin GPU die/transistor/SM/Tensor Core counts, HBM4 288GB@22TB/s, 50 PFLOPS NVFP4, FP32/BF16/FP16 throughput multipliers vs. Blackwell.
  5. officialNVIDIA. NVIDIA H100 Tensor Core GPU Architecture. Whitepaper v1.02, 2022. https://www.hpctech.co.jp/assets/images/info/catalog/pdf/gtc22-whitepaper-hopper_v1.02.pdf. Accessed 2026-09-17. Used for: GH100 die/SM counts (Fig. 6, p.18-19), SM partition/Tensor Core microarchitecture (Fig. 7, p.21), FP8 E4M3/E5M2 formats (p.23-24), Transformer Engine mechanism (Fig. 25, p.46), fourth-generation NVLink/NVSwitch specifics (p.47-49), DGX SuperPOD comparison (Fig. 26, p.49), H100-vs-A100 "30x" inference claim (p.9, p.19). NVIDIA's own hosting (nvdam.widen.net) is access-gated; this is a third-party-hosted copy of the identical official document, verified against NVIDIA's public GTC 2022 release.
  6. officialNVIDIA. H200 GPU (product page). 2026. https://www.nvidia.com/en-us/data-center/h200/. Accessed 2026-09-17. Used for: H200 HBM3e capacity/bandwidth, FP8 Tensor Core TFLOPS (unchanged from H100), H200-vs-H100 inference speedup claims.
  7. blogNVIDIA. Inside NVIDIA Blackwell Ultra: The Chip Powering the AI Factory Era. NVIDIA Technical Blog, 2025-08-22. https://developer.nvidia.com/blog/inside-nvidia-blackwell-ultra-the-chip-powering-the-ai-factory-era/. Accessed 2026-09-17. Used for: NV-HBI interconnect name and 10TB/s bandwidth, "reticle-sized dies" framing, Blackwell Ultra's 160 SM/8 GPC/640 Tensor Core/15 PFLOPS dense NVFP4 configuration.
  8. officialNVIDIA. The Engine Behind AI Factories — NVIDIA Blackwell Architecture (architecture page). 2026. https://www.nvidia.com/en-us/data-center/technologies/blackwell-architecture/. Accessed 2026-09-17. Used for: Blackwell 208B transistor/dual-die/10TB/s die-to-die figures, NVLink5/NVSwitch 130TB/s NVL72 domain and 1.8TB/s multi-server figures, SHARP FP8 efficiency, Blackwell Ultra 1.5x/2x claims, native FP4 framing.
  9. officialNVIDIA. DGX B200 (product page). 2026. https://www.nvidia.com/en-us/data-center/dgx-b200/. Accessed 2026-09-17. Used for: 8-GPU DGX B200 system totals (1,440GB HBM3e @ 64TB/s, FP4 144/72 PFLOPS sparse/dense, FP8 72 PFLOPS sparse, 14.4TB/s NVLink, ~14.3kW system power) and the explicit "dense = ½ sparse" convention, divided to per-GPU figures in Table 1.
  10. repoNVIDIA. NVFP4 (Transformer Engine documentation). https://nvidia.github.io/TransformerEngine/features/low_precision_training/nvfp4/nvfp4.html. Accessed 2026-09-17. Used for: NVFP4 E2M1 bit layout, ±6 maximum magnitude, 16-element block scaling with FP8 E4M3 block scale plus FP32 global scale, default 2D/1D scaling recipe.
  11. blogNVIDIA. NVIDIA GB200 NVL72 Delivers Trillion-Parameter LLM Training and Real-Time Inference. NVIDIA Technical Blog, 2024. https://developer.nvidia.com/blog/nvidia-gb200-nvl72-delivers-trillion-parameter-llm-training-and-real-time-inference/. Accessed 2026-09-17. Used for: GPT-MoE-1.8T 150-vs-3.4 tokens/sec/GPU and "30x"/"4x" claims, per-GPU 1.8TB/s NVLink figure, "30TB unified memory"/"over 1PB/s" rack claims.
  12. blogNVIDIA. NVIDIA Blackwell Delivers Massive Performance Leaps in MLPerf Inference v5.0. NVIDIA Developer Blog, 2025. https://developer.nvidia.com/blog/nvidia-blackwell-delivers-massive-performance-leaps-in-mlperf-inference-v5-0/. Accessed 2026-09-17. Used for: Llama 3.1 405B 3.4x per-GPU and "9x more GPUs"/"30x" decomposition, B200-vs-H200 Llama 2 70B and Mixtral throughput tables.
  13. analysisSemiAnalysis. Nvidia Blackwell Perf TCO Analysis — B100 vs B200 vs GB200NVL72. 2024-04-10. https://newsletter.semianalysis.com/p/nvidia-blackwell-perf-tco-analysis. Accessed 2026-09-17. Used for: independent recomputation of the GB200-vs-H200 gain (~18x at matched precision, under 8x at a different token profile) and the quantization/parallelism-scheme critique of NVIDIA's footnoted benchmark configuration.
  14. paperMicikevicius, P., Stosic, D., Burgess, N., Cornea, M., Dubey, P., Grisenthwaite, R., Ha, S., Heinecke, A., Judd, P., Kamalu, J., Mellempudi, N., Oberman, S., Shoeybi, M., Siu, M., Wu, H. FP8 Formats for Deep Learning. arXiv:2209.05433, 2022 (v2, 2022-09-29). https://arxiv.org/abs/2209.05433. Accessed 2026-09-17. Used for: E4M3/E5M2 dynamic-range-vs-precision tradeoff and E4M3's extended range via forgoing infinities/limiting NaN encodings.
  15. blogNVIDIA. NVIDIA Vera Rubin POD: Seven Chips, Five Rack-Scale Systems, One AI Supercomputer. NVIDIA Developer Blog, 2026. https://developer.nvidia.com/blog/nvidia-vera-rubin-pod-seven-chips-five-rack-scale-systems-one-ai-supercomputer/. Accessed 2026-09-17. Used for: per-GPU/per-rack NVLink6 figures, Vera CPU rack detail, ConnectX-9/ BlueField-4/Spectrum-6 component specifics.
  16. officialNVIDIA. NVIDIA Vera Rubin Platform (technology page). 2026. https://www.nvidia.com/en-us/data-center/technologies/rubin/. Accessed 2026-09-17. Used for: Vera Rubin NVL72 composition, Vera CPU rack (256 CPUs, 22,500+ sandbox environments), Vera Rubin NVL4 comparative claims, "now in full production" status statement.
  17. officialNVIDIA. NVIDIA Unveils Rubin CPX: A New Class of GPU Designed for Massive-Context Inference. NVIDIA Newsroom, 2025-09-09. https://nvidianews.nvidia.com/news/nvidia-unveils-rubin-cpx-a-new-class-of-gpu-designed-for-massive-context-inference. Accessed 2026-09-17. Used for: Rubin CPX 128GB GDDR7/30 PFLOPS NVFP4 specs, 3x attention claim, Vera Rubin NVL144 CPX rack specs, token-revenue economics claim.