~/best-gpu-for/mixtral-8x22b
Mistral provider

Best GPU for Mixtral 8x22B

Ranked recommendations from the 42-GPU catalog for running Mixtral 8x22B (141B params). Top picks by quality, then by cost.

Top picks.

$ ./rank --by quality
Tightest budget
H100 80GB
80GB · runs at Q3_K_M
61 GB weights · 4.8 GB headroom
Best quality
M3 Ultra 512
512GB · runs at FP16/BF16
smallest card supporting the best available quant
Apple Silicon
M3 Max 96
96GB · runs at AWQ 4-bit
smallest M-series that fits
Datacenter (FP16 ambition)
B200
192GB · runs at Q8_0
best-quality datacenter card that fits

Full ranked list.

$ ./vrambudget --rank mixtral-8x22b
GPUVRAMBest quantWeights at 15% safetyFit
appleM3 Ultra 512512GBFP16/BF16
282fitsnvidia2× H100 NVL188GBQ8_0
150fitsappleM2 Ultra 192192GBQ8_0
150fitsnvidiaB200192GBQ8_0
150fitsamdMI300X192GBQ8_0
150fitsappleM4 Max 128128GBQ5_K_M
97fitsappleM5 Max 128128GBQ5_K_M
97fitsnvidiaDGX Spark128GBQ5_K_M
97fitsintelGaudi 3128GBQ5_K_M
97fitsnvidiaH200141GBQ5_K_M
97fitsappleM3 Max 9696GBAWQ 4-BIT
75fitsnvidiaRTX Pro 600096GBAWQ 4-BIT
75fitsnvidiaH100 80GB80GBQ3_K_M
61fitsnvidiaRTX 30708GBQ4_K_M
79overnvidiaRTX 40608GBQ4_K_M
79overnvidiaRTX 308010GBQ4_K_M
79overnvidiaRTX 3060 12GB12GBQ4_K_M
79overnvidiaRTX 3080 Ti12GBQ4_K_M
79over

Next steps.

$ ./next
$ checkCan I run Mixtral 8x22B?see all quants →full quant matrix$ tuneTune the mathopen calculator →/calc?model=mixtral-8x22b

Discussion.

$ gh discussion list

// sign in with github to leave a comment. threads live in the repo's discussions tab.