~/best-gpu-for/qwen-2-5-72b
Alibaba provider

Best GPU for Qwen 2.5 72B

Ranked recommendations from the 42-GPU catalog for running Qwen 2.5 72B (72.7B params). Top picks by quality, then by cost.

Top picks.

$ ./rank --by quality
Tightest budget
RTX A6000
48GB · runs at Q3_K_M
31 GB weights · 7.3 GB headroom
Best quality
2× H100 NVL
188GB · runs at FP16/BF16
smallest card supporting the best available quant
Apple Silicon
M2 Max 64
64GB · runs at Q5_K_M
smallest M-series that fits
Datacenter (FP16 ambition)
B200
192GB · runs at FP16/BF16
best-quality datacenter card that fits

Full ranked list.

$ ./vrambudget --rank qwen-2-5-72b
GPUVRAMBest quantWeights at 15% safetyFit
nvidia2× H100 NVL188GBFP16/BF16
145fitsappleM2 Ultra 192192GBFP16/BF16
145fitsnvidiaB200192GBFP16/BF16
145fitsamdMI300X192GBFP16/BF16
145fitsappleM3 Ultra 512512GBFP16/BF16
145fitsappleM4 Max 128128GBQ8_0
77fitsappleM5 Max 128128GBQ8_0
77fitsnvidiaDGX Spark128GBQ8_0
77fitsintelGaudi 3128GBQ8_0
77fitsnvidiaH200141GBQ8_0
77fitsappleM3 Max 9696GBFP8/INT8
73fitsnvidiaRTX Pro 600096GBFP8/INT8
73fitsnvidiaH100 80GB80GBQ6_K
60fitsappleM2 Max 6464GBQ5_K_M
50fitsappleM3 Max 6464GBQ5_K_M
50fitsappleM4 Pro 6464GBQ5_K_M
50fitsappleM5 Pro 6464GBQ5_K_M
50fitsnvidiaRTX A600048GBQ3_K_M
31fits

Next steps.

$ ./next
$ checkCan I run Qwen 2.5 72B?see all quants →full quant matrix$ tuneTune the mathopen calculator →/calc?model=qwen-2-5-72b

Discussion.

$ gh discussion list

// sign in with github to leave a comment. threads live in the repo's discussions tab.