~/best-gpu-for/mixtral-8x7b
Mistral provider

Best GPU for Mixtral 8x7B

Ranked recommendations from the 42-GPU catalog for running Mixtral 8x7B (46.7B params). Top picks by quality, then by cost.

Top picks.

$ ./rank --by quality
Tightest budget
RTX 5090
32GB · runs at AWQ 4-bit
25 GB weights · 0.54 GB headroom
Best quality
M4 Max 128
128GB · runs at FP16/BF16
smallest card supporting the best available quant
Apple Silicon
M2 Max 64
64GB · runs at Q8_0
smallest M-series that fits
Datacenter (FP16 ambition)
H200
141GB · runs at FP16/BF16
best-quality datacenter card that fits

Full ranked list.

$ ./vrambudget --rank mixtral-8x7b
GPUVRAMBest quantWeights at 15% safetyFit
appleM4 Max 128128GBFP16/BF16
93fitsappleM5 Max 128128GBFP16/BF16
93fitsnvidiaDGX Spark128GBFP16/BF16
93fitsintelGaudi 3128GBFP16/BF16
93fitsnvidiaH200141GBFP16/BF16
93fitsnvidia2× H100 NVL188GBFP16/BF16
93fitsappleM2 Ultra 192192GBFP16/BF16
93fitsnvidiaB200192GBFP16/BF16
93fitsamdMI300X192GBFP16/BF16
93fitsappleM3 Ultra 512512GBFP16/BF16
93fitsappleM2 Max 6464GBQ8_0
50fitsappleM3 Max 6464GBQ8_0
50fitsappleM4 Pro 6464GBQ8_0
50fitsappleM5 Pro 6464GBQ8_0
50fitsnvidiaH100 80GB80GBQ8_0
50fitsappleM3 Max 9696GBQ8_0
50fitsnvidiaRTX Pro 600096GBQ8_0
50fitsnvidiaRTX A600048GBQ5_K_M
32fits

Next steps.

$ ./next
$ checkCan I run Mixtral 8x7B?see all quants →full quant matrix$ tuneTune the mathopen calculator →/calc?model=mixtral-8x7b

Discussion.

$ gh discussion list

// sign in with github to leave a comment. threads live in the repo's discussions tab.