~/can-i-run/mixtral-8x22b
Mistral provider

Can I run Mixtral 8x22B?

Short answer: yes, on a M3 Ultra 512 (512GB) at FP16/BF16. Long answer below.

mixtral:8x22bLM StudiovLLMMLXoMLX

The math, in one paragraph.

$ ./vrambudget --explain mixtral-8x22b

Mixtral 8x22B has 141B parameters (MoE: 39B active per forward pass, but all 141B must fit in memory). At FP16 that's 282 GB of raw weights. Quantization shrinks that, but you also need budget for the KV cache (definition), framework overhead, and safety headroom. The rule of thumb: real usable budget on a card is roughly its nameplate VRAM minus 25%. That's how the table below was computed.

What hardware actually fits.

$ grep "fits" gpus.json
FP16/BF16
282GB
1 GPU fits
M3 Ultra 512512GB
Q8_0
150GB
5 GPUs fit
2× H100 NVL188GBM2 Ultra 192192GBB200192GBMI300X192GB+ 1 more
Q5_K_M
97GB
10 GPUs fit
M4 Max 128128GBM5 Max 128128GBDGX Spark128GBGaudi 3128GB+ 6 more
Q4_K_M
79GB
10 GPUs fit
M4 Max 128128GBM5 Max 128128GBDGX Spark128GBGaudi 3128GB+ 6 more
Q3_K_M
61GB
13 GPUs fit
H100 80GB80GBM3 Max 9696GBRTX Pro 600096GBM4 Max 128128GB+ 9 more

Pick your path.

$ ls strategies/
Tightest budget

Smallest GPU that fits Mixtral 8x22B at any quant: M3 Ultra 512 at FP16/BF16.

Reference quality (FP16)

Lossless inference needs 282 GB. Pick from 1 cards.

Best quality on a 24GB card

None of the showcase quants fit on a 24GB card. Step up.

Tune the math yourself

Open the calculator pre-tuned for Mixtral 8x22B: ↗ /calc?model=mixtral-8x22b

See the full model page.

$ ./open
$ openMixtral 8x22B141BMistral · ctx 64K$ rankBest GPU for Mixtral 8x22Bsee the ranking →ranked by fit quality

Discussion.

$ gh discussion list

// sign in with github to leave a comment. threads live in the repo's discussions tab.