~/model/huggingfacedeepseek-ai/DeepSeek-V3
DeepSeek

DeepSeek V3

671B MoE, 37B active. GPT-4 class on a fraction of the inference cost. Multi-node serving at FP16.

Parameters
671B
Family
DeepSeek
Context
128K tokens
FP16 weights
1342GB
// where you can run it
deepseek-v3LM StudiovLLMMLXoMLX
// hugging face stats (cached daily)
1.2M downloads · 4K likes · updated 1 year ago

What you need to run this.

$ ./vrambudget --model deepseek-v3 --by quant
FP161342 GB
M3 Ultra 512512 GB · overM2 Ultra 192192 GB · overB200192 GB · overMI300X192 GB · over
Q8_0713 GB
M3 Ultra 512512 GB · overM2 Ultra 192192 GB · overB200192 GB · overMI300X192 GB · over
Q6_K550 GB
M3 Ultra 512512 GB · overM2 Ultra 192192 GB · overB200192 GB · overMI300X192 GB · over
Q5_K_M461 GB
M3 Ultra 512512 GB · tightM2 Ultra 192192 GB · overB200192 GB · overMI300X192 GB · over
Q4_K_M377 GB
M3 Ultra 512512 GB · fitsM2 Ultra 192192 GB · overB200192 GB · overMI300X192 GB · over

// budgets shown at ctx 8K, concurrency 1, 15% safety headroom. Tune in the calculator →

Alternatives at this size.

$ grep --params similar catalog.json
$ similarDeepSeek R1671BDeepSeek$ similarLlama 3.1 405B405BMeta$ similarMixtral 8x22B141BMistral$ similargpt-oss 120B117BOpenAI

Discussion.

$ gh discussion list

// sign in with github to leave a comment. threads live in the repo's discussions tab.