~/can-i-run/gemma-4-26b
Google provider

Can I run Gemma 4 26B A4B?

Short answer: yes, on a H100 80GB (80GB) at FP16/BF16. Long answer below.

OllamaLM StudiovLLMMLXoMLX

The math, in one paragraph.

$ ./vrambudget --explain gemma-4-26b

Gemma 4 26B A4B has 26B parameters (MoE: 4B active per forward pass, but all 26B must fit in memory). At FP16 that's 52 GB of raw weights. Quantization shrinks that, but you also need budget for the KV cache (definition), framework overhead, and safety headroom. The rule of thumb: real usable budget on a card is roughly its nameplate VRAM minus 25%. That's how the table below was computed.

What hardware actually fits.

$ grep "fits" gpus.json
FP16/BF16
52GB
13 GPUs fit
H100 80GB80GBM3 Max 9696GBRTX Pro 600096GBM4 Max 128128GB+ 9 more
Q8_0
28GB
21 GPUs fit
RTX A600048GBRTX 6000 Ada48GBL40S48GBRadeon Pro W790048GB+ 17 more
Q5_K_M
18GB
27 GPUs fit
RTX 309024GBRTX 3090 Ti24GBRTX 409024GBRX 7900 XTX24GB+ 23 more
Q4_K_M
15GB
27 GPUs fit
RTX 309024GBRTX 3090 Ti24GBRTX 409024GBRX 7900 XTX24GB+ 23 more
Q3_K_M
11GB
33 GPUs fit
RTX 4060 Ti 16GB16GBRTX 4070 Ti Super16GBRTX 408016GBRTX 4080 Super16GB+ 29 more

Pick your path.

$ ls strategies/
Tightest budget

Smallest GPU that fits Gemma 4 26B A4B at any quant: H100 80GB at FP16/BF16.

Reference quality (FP16)

Lossless inference needs 52 GB. Pick from 13 cards.

Best quality on a 24GB card

Q5_K_M fits comfortably (18 GB weights).

Tune the math yourself

Open the calculator pre-tuned for Gemma 4 26B A4B: ↗ /calc?model=gemma-4-26b

See the full model page.

$ ./open
$ openGemma 4 26B A4B26BGoogle · ctx 128K$ rankBest GPU for Gemma 4 26B A4Bsee the ranking →ranked by fit quality

Discussion.

$ gh discussion list

// sign in with github to leave a comment. threads live in the repo's discussions tab.