~/model/huggingfaceQwen/Qwen2.5-72B-Instruct
Alibaba provider

Qwen 2.5 72B

Alibaba's 70B-class flagship. Comparable footprint to Llama 70B; different inflections on benchmarks.

Parameters
72.7B
Family
Alibaba
Context
128K tokens
FP16 weights
145GB
// where you can run it
qwen2.5:72bLM StudiovLLMMLXoMLX
// hugging face stats (cached daily)
883K downloads · 940 likes · license: other · updated 1 year ago

What you need to run this.

$ ./vrambudget --model qwen-2-5-72b --by quant
FP16145 GB
2× H100 NVL188 GB · fitsM2 Ultra 192192 GB · fitsB200192 GB · fitsMI300X192 GB · fits
Q8_077 GB
M3 Max 9696 GB · fitsRTX Pro 600096 GB · fitsM4 Max 128128 GB · fitsM5 Max 128128 GB · fits
Q6_K60 GB
H100 80GB80 GB · fitsM3 Max 9696 GB · fitsRTX Pro 600096 GB · fitsM4 Max 128128 GB · fits
Q5_K_M50 GB
M2 Max 6464 GB · fitsM3 Max 6464 GB · fitsM4 Pro 6464 GB · fitsM5 Pro 6464 GB · fits
Q4_K_M41 GB
M2 Max 6464 GB · fitsM3 Max 6464 GB · fitsM4 Pro 6464 GB · fitsM5 Pro 6464 GB · fits

// budgets shown at ctx 8K, concurrency 1, 15% safety headroom. Tune in the calculator →

Alternatives at this size.

$ grep --params similar catalog.json
$ similarLlama 3.3 70B70.6BMeta$ similarMixtral 8x7B46.7BMistral$ similarCommand R+104BCohere$ similarQwen 3.6 35B A3B35BAlibaba

Discussion.

$ gh discussion list

// sign in with github to leave a comment. threads live in the repo's discussions tab.