~/gpu/h100 vs h200

H100 80GB manufacturerH100 80GBvsH200 manufacturerH200

Head-to-head for local LLM inference. The honest comparison: VRAM, bandwidth, compute, and which of the 30 catalog models actually fit on each.

The specs.

$ diff specs h100 h200
Stat
h100
h200
Δ
VRAM
80 GB
141 GB
+76%
Memory bandwidth
3,350 GB/s
4,800 GB/s
+43%
FP16 compute
989 TFLOPS
989 TFLOPS
0%
Weights budget at 8K ctx
63 GB
113 GB
+79%

Model fit difference.

$ models that change with the card
Fits on both
27of 30
Only on h100
0
Only on h200
0

// showing 12 of 30 models; differing fits first

Model
h100
h200
fitsFP16/BF16
fitsFP16/BF16
fitsFP16/BF16
fitsFP16/BF16
fitsFP16/BF16
fitsFP16/BF16
fitsQ6_K
fitsQ8_0
overQ4_K_M
overQ4_K_M
fitsFP16/BF16
fitsFP16/BF16
fitsQ8_0
fitsFP16/BF16
fitsQ8_0
fitsFP16/BF16
fitsQ6_K
fitsQ8_0
fitsFP16/BF16
fitsFP16/BF16
fitsFP16/BF16
fitsFP16/BF16
fitsFP16/BF16
fitsFP16/BF16

Which one wins for…

$ ./recommend --by-workload
More VRAM headroom

H200 has 61 GB more.

Faster decode (bandwidth)

H200 by +43%.

Faster prefill (compute)

Tied at 989 TFLOPS.

Catalog models that fit

Tied: 27 of 30 fit on each.

Drill into either card.

$ ./vrambudget --gpu

Discussion.

$ gh discussion list

// sign in with github to leave a comment. threads live in the repo's discussions tab.