~/model/huggingface01-ai/Yi-34B-Chat
01.AI

Yi 34B

01.AI's 34B. Bilingual EN/ZH; long-context variants up to 200K available.

Parameters
34B
Family
01.AI
Context
32K tokens
FP16 weights
68GB
// where you can run it
yi:34bLM StudiovLLMMLXoMLX
// hugging face stats (cached daily)
74K downloads · 356 likes · license: apache-2.0 · updated 1 year ago

What you need to run this.

$ ./vrambudget --model yi-34b --by quant
FP1668 GB
M3 Max 9696 GB · fitsRTX Pro 600096 GB · fitsM4 Max 128128 GB · fitsM5 Max 128128 GB · fits
Q8_036 GB
RTX A600048 GB · fitsRTX 6000 Ada48 GB · fitsL40S48 GB · fitsRadeon Pro W790048 GB · fits
Q6_K28 GB
RTX A600048 GB · fitsRTX 6000 Ada48 GB · fitsL40S48 GB · fitsRadeon Pro W790048 GB · fits
Q5_K_M23 GB
RTX 509032 GB · fitsRTX A600048 GB · fitsRTX 6000 Ada48 GB · fitsL40S48 GB · fits
Q4_K_M19 GB
RTX 509032 GB · fitsRTX A600048 GB · fitsRTX 6000 Ada48 GB · fitsL40S48 GB · fits

// budgets shown at ctx 8K, concurrency 1, 15% safety headroom. Tune in the calculator →

Alternatives at this size.

$ grep --params similar catalog.json
$ similarQwen 3.6 35B A3B35BAlibaba$ similarQwen 2.5 32B32.5BAlibaba$ similarQwen 2.5 Coder 32B32.5BAlibaba$ similarQwen3 30B A3B30.5BAlibaba

Discussion.

$ gh discussion list

// sign in with github to leave a comment. threads live in the repo's discussions tab.