~/best-gpu-for/llama-3-2-1b
Meta provider

Best GPU for Llama 3.2 1B

Ranked recommendations from the 42-GPU catalog for running Llama 3.2 1B (1.23B params). Top picks by quality, then by cost.

Top picks.

$ ./rank --by quality
Tightest budget
RTX 3070
8GB · runs at FP16/BF16
2.5 GB weights · 3.1 GB headroom
Best quality
RTX 3070
8GB · runs at FP16/BF16
smallest card supporting the best available quant
Apple Silicon
M2 Max 64
64GB · runs at FP16/BF16
smallest M-series that fits
Datacenter (FP16 ambition)
H100 80GB
80GB · runs at FP16/BF16
best-quality datacenter card that fits

Full ranked list.

$ ./vrambudget --rank llama-3-2-1b
GPUVRAMBest quantWeights at 15% safetyFit
nvidiaRTX 30708GBFP16/BF16
2.5fitsnvidiaRTX 40608GBFP16/BF16
2.5fitsnvidiaRTX 308010GBFP16/BF16
2.5fitsnvidiaRTX 3060 12GB12GBFP16/BF16
2.5fitsnvidiaRTX 3080 Ti12GBFP16/BF16
2.5fitsnvidiaRTX 407012GBFP16/BF16
2.5fitsnvidiaRTX 4070 Super12GBFP16/BF16
2.5fitsnvidiaRTX 507012GBFP16/BF16
2.5fitsintelArc B58012GBFP16/BF16
2.5fitsnvidiaRTX 4060 Ti 16GB16GBFP16/BF16
2.5fitsnvidiaRTX 4070 Ti Super16GBFP16/BF16
2.5fitsnvidiaRTX 408016GBFP16/BF16
2.5fitsnvidiaRTX 4080 Super16GBFP16/BF16
2.5fitsnvidiaRTX 5070 Ti16GBFP16/BF16
2.5fitsnvidiaRTX 508016GBFP16/BF16
2.5fitsnvidiaRTX 309024GBFP16/BF16
2.5fitsnvidiaRTX 3090 Ti24GBFP16/BF16
2.5fitsnvidiaRTX 409024GBFP16/BF16
2.5fits

Next steps.

$ ./next
$ checkCan I run Llama 3.2 1B?see all quants →full quant matrix$ tuneTune the mathopen calculator →/calc?model=llama-3-2-1b

Discussion.

$ gh discussion list

// sign in with github to leave a comment. threads live in the repo's discussions tab.