~/model/huggingfacemicrosoft/Phi-4-mini-instruct
Microsoft

Phi-4 Mini

3.8B Phi-4 distill. Strong reasoning at edge sizes; 128K context.

Parameters
3.8B
Family
Microsoft
Context
128K tokens
FP16 weights
7.6GB
// where you can run it
phi4-miniLM StudiovLLMMLXoMLX
// hugging face stats (cached daily)
1.6M downloads · 739 likes · license: mit · updated 5 months ago

What you need to run this.

$ ./vrambudget --model phi-4-mini --by quant
FP167.6 GB
RTX 3060 12GB12 GB · fitsRTX 3080 Ti12 GB · fitsRTX 407012 GB · fitsRTX 4070 Super12 GB · fits
Q8_04.0 GB
RTX 30708 GB · fitsRTX 40608 GB · fitsRTX 308010 GB · fitsRTX 3060 12GB12 GB · fits
Q6_K3.1 GB
RTX 30708 GB · fitsRTX 40608 GB · fitsRTX 308010 GB · fitsRTX 3060 12GB12 GB · fits
Q5_K_M2.6 GB
RTX 30708 GB · fitsRTX 40608 GB · fitsRTX 308010 GB · fitsRTX 3060 12GB12 GB · fits
Q4_K_M2.1 GB
RTX 30708 GB · fitsRTX 40608 GB · fitsRTX 308010 GB · fitsRTX 3060 12GB12 GB · fits

// budgets shown at ctx 8K, concurrency 1, 15% safety headroom. Tune in the calculator →

Alternatives at this size.

$ grep --params similar catalog.json
$ similarGemma 4 E4B4BGoogle$ similarLlama 3.2 3B3.21BMeta$ similarLlama 3.2 1B1.23BMeta$ similarQwen 2.5 7B7BAlibaba

Discussion.

$ gh discussion list

// sign in with github to leave a comment. threads live in the repo's discussions tab.