~/can-i-run/qwen-3-5-9b
Alibaba provider

Can I run Qwen 3.5 9B?

Short answer: yes, on a RTX 3090 (24GB) at FP16/BF16. Long answer below.

OllamaLM StudiovLLMMLXoMLX

The math, in one paragraph.

$ ./vrambudget --explain qwen-3-5-9b

Qwen 3.5 9B has 9B parameters. At FP16 that's 18 GB of raw weights. Quantization shrinks that, but you also need budget for the KV cache (definition), framework overhead, and safety headroom. The rule of thumb: real usable budget on a card is roughly its nameplate VRAM minus 25%. That's how the table below was computed.

What hardware actually fits.

$ grep "fits" gpus.json
FP16/BF16
18GB
27 GPUs fit
RTX 309024GBRTX 3090 Ti24GBRTX 409024GBRX 7900 XTX24GB+ 23 more
Q8_0
9.6GB
33 GPUs fit
RTX 4060 Ti 16GB16GBRTX 4070 Ti Super16GBRTX 408016GBRTX 4080 Super16GB+ 29 more
Q5_K_M
6.2GB
40 GPUs fit
RTX 308010GBRTX 3060 12GB12GBRTX 3080 Ti12GBRTX 407012GB+ 36 more
Q4_K_M
5.1GB
42 GPUs fit
RTX 30708GBRTX 40608GBRTX 308010GBRTX 3060 12GB12GB+ 38 more
Q3_K_M
3.9GB
42 GPUs fit
RTX 30708GBRTX 40608GBRTX 308010GBRTX 3060 12GB12GB+ 38 more

Pick your path.

$ ls strategies/
Tightest budget

Smallest GPU that fits Qwen 3.5 9B at any quant: RTX 3090 at FP16/BF16.

Reference quality (FP16)

Lossless inference needs 18 GB. Pick from 27 cards.

Best quality on a 24GB card

FP16/BF16 fits comfortably (18 GB weights).

Tune the math yourself

Open the calculator pre-tuned for Qwen 3.5 9B: ↗ /calc?model=qwen-3-5-9b

See the full model page.

$ ./open
$ openQwen 3.5 9B9BAlibaba · ctx 128K$ rankBest GPU for Qwen 3.5 9Bsee the ranking →ranked by fit quality

Discussion.

$ gh discussion list

// sign in with github to leave a comment. threads live in the repo's discussions tab.