~/gpu/h100 vs h100-nvl-2x

H100 80GB manufacturerH100 80GBvs2× H100 NVL manufacturer2× H100 NVL

Head-to-head for local LLM inference. The honest comparison: VRAM, bandwidth, compute, and which of the 30 catalog models actually fit on each.

The specs.

$ diff specs h100 h100-nvl-2x
Stat
h100
h100-nvl-2x
Δ
VRAM
80 GB
188 GB
+135%
Memory bandwidth
3,350 GB/s
3,938 GB/s
+18%
FP16 compute
989 TFLOPS
1979 TFLOPS
+100%
Weights budget at 8K ctx
63 GB
151 GB
+140%

Model fit difference.

$ models that change with the card
Fits on both
27of 30
Only on h100
0
Only on h100-nvl-2x
0

// showing 12 of 30 models; differing fits first

Model
h100
h100-nvl-2x
Llama 3.2 1B1.23B
fitsFP16/BF16
fitsFP16/BF16
Llama 3.2 3B3.21B
fitsFP16/BF16
fitsFP16/BF16
Llama 3.1 8B8B
fitsFP16/BF16
fitsFP16/BF16
Llama 3.3 70B70.6B
fitsQ6_K
fitsFP16/BF16
Llama 3.1 405B405B
overQ4_K_M
overQ4_K_M
Qwen 2.5 7B7B
fitsFP16/BF16
fitsFP16/BF16
Qwen 2.5 32B32.5B
fitsQ8_0
fitsFP16/BF16
Qwen 2.5 Coder 32B32.5B
fitsQ8_0
fitsFP16/BF16
Qwen 2.5 72B72.7B
fitsQ6_K
fitsFP16/BF16
Qwen3 30B A3B30.5B
fitsFP16/BF16
fitsFP16/BF16
Qwen 3.5 9B9B
fitsFP16/BF16
fitsFP16/BF16
Qwen 3.6 27B27B
fitsFP16/BF16
fitsFP16/BF16

Which one wins for…

$ ./recommend --by-workload
More VRAM headroom

2× H100 NVL has 108 GB more.

Faster decode (bandwidth)

2× H100 NVL by +18%.

Faster prefill (compute)

2× H100 NVL by +100% TFLOPS.

Catalog models that fit

Tied: 27 of 30 fit on each.

Drill into either card.

$ ./vrambudget --gpu
$ openH100 80GB80GBhopper$ open2× H100 NVL188GBmulti-gpu

Discussion.

$ gh discussion list

// sign in with github to leave a comment. threads live in the repo's discussions tab.