LLM VRAM Calculator

How the estimate works

The calculator applies the method from our GPU sizing guide layer by layer, which is what lets it handle models that mix attention types. All sizes are decimal gigabytes (1 GB = 10⁹ bytes), the unit that guide uses, and GPU capacities are compared as the vendors print them.

1. Weights. Total parameters × bits per weight ÷ 8. A mixture-of-experts model keeps every expert loaded, so the total count applies even though each token touches only the active share.
2. KV cache per token. In each layer, standard and grouped-query attention store one key and one value per KV head: 2 × KV heads × head dimension values. DeepSeek’s multi-head latent attention stores a single compressed vector instead, kv_lora_rank + qk_rope_head_dim values, which is 576 for DeepSeek-V3.1. Multiply by 2 bytes for a 16-bit cache or 1 byte for FP8, then sum over the layers.
3. Windowed layers. A sliding-window layer attends only to its most recent tokens and a chunked layer only to its current chunk, so both count min(context, window). vLLM documents that its hybrid KV cache manager reserves just the latest window on sliding-window layers, and it lists Llama 4’s chunked layout among the hybrids it manages; llama.cpp keeps a window-sized cache unless started with --swa-full. If your engine caches these layers at full length, switch on the Advanced toggle.
4. KV cache in total. KV cache per sequence × concurrent sequences.
5. Concurrent sequences. Named users × peak-active share, rounded up: 20% by default, 10% for light use. A team of fewer than ten counts everyone. You can also enter sequences directly.
6. Memory to provision. (Weights + KV cache + other resident models) ÷ usable share. The default share of 0.90 holds a tenth back for activations and the runtime.
7. GPU count. The smallest of 1, 2, 4 or 8 GPUs whose combined memory covers that figure, then whole 8-GPU nodes. Below 90% of the configuration’s memory the result reads “fits”, from 90% to 100% “tight”, and above that “does not fit”.
8. Decode ceiling. Memory bandwidth ÷ (bytes of active weights + one sequence’s KV cache). Each generated token reads both once, so this caps single-stream speed from above; real throughput lands lower.

To check the arithmetic by hand, take Qwen3-8B’s config.json: 2 × 36 layers × 8 KV heads × 128 dimensions × 2 bytes comes to 147,456 bytes per token, so one 8,192-token conversation holds 1.2 GB of cache.

Bits per weight by format

Quantized formats store a scale next to each block of weights, so their real cost sits above the nominal bit width. The GGUF figures are the averages llama.cpp measured on Llama 3.1 8B. For what each format does to accuracy, see how quantization formats differ.

From headcount to GPUs

Capacity requests tend to arrive as a number of employees rather than a model name. Only a slice of them has a request in flight at the busiest moment, so the calculator converts named users into concurrent sequences first: 20% unless you choose 10%, and everyone in a team smaller than ten. Our sizing walkthrough for 5, 50 and 200 users explains those assumptions; the table recomputes its scenarios with this page’s presets.

The guide rounds its 70B model to 70 billion parameters, while the Llama 3.3 preset carries the published 70.55 billion. That is why the two 200-user FP8 rows read 198 GB and 138 GB here, about 1 GB above the guide’s 197 and 137 GB.

A count that only just fits leaves no room for a failed GPU or a node taken down for patching, so production plans usually add one more server than the arithmetic asks for. The reference architecture lists GPU server classes by model size, and the server buying guide covers power, cooling and the software stack around these GPUs.

What the calculator leaves out

A memory estimate tells you whether a plan is plausible before you buy, not that a model will load on the first try. These effects sit inside the reserve or on top of it:

• Activations and prefill buffers. Processing long prompts in large batches needs working memory that grows with the batch’s token count; the reserve covers ordinary batches, not every prefill setting.
• Fragmentation. Paged KV caches hand out memory in fixed-size blocks, so a little is stranded at the end of each sequence.
• CUDA graphs and runtime state. The CUDA or ROCm context, communication buffers and captured CUDA graphs claim memory on every GPU before the first request arrives.
• Multi-GPU and multi-node overhead. Tensor parallelism shards the weights but adds communication buffers, and pipeline stages spread across nodes bring their own buffers and slower links.
• Speculative decoding. A draft model or multi-token-prediction head adds weights and a KV cache of its own; enter it under other resident models.
• Shared prefixes. Prefix caching can store a long system prompt once for many conversations; the calculator assumes nothing is shared.
• Hybrid linear-attention models. Layers built on linear attention or state-space blocks keep a fixed-size state per sequence rather than a growing cache. Model only the full-attention layers as a custom entry and add the state as another resident model.
• Speed. The decode ceiling ignores compute, batching and interconnects, and prefill is limited by compute rather than bandwidth. For mixed-precision checkpoints it also spreads the file’s average bits over the active parameters, so it can run high. Measure throughput with your own prompts.

Sources (verified October 2026)

Model configurations and cards

• Qwen3-8B config.json
• Qwen3-8B model card
• Qwen3-32B config.json
• Qwen3-32B model card
• Devstral Small 2 config.json
• Devstral Small 2 (24B) model card
• Gemma 4 31B config.json
• Gemma 4 31B model card
• Llama 3.3 70B model card
• The Llama 3 Herd of Models (Table 3)
• gpt-oss-20b config.json
• gpt-oss-20b model card
• gpt-oss-120b config.json
• gpt-oss-120b model card
• Llama 4 Scout model card
• Llama 4 Scout config.json (public copy of the gated file)
• Meta reference code: NoPE and chunked layers
• Llama 4 Maverick model card
• Llama 4 Maverick config.json (public copy of the gated file)
• DeepSeek-V3.1 config.json
• DeepSeek-V3.1 (671B) model card
• DeepSeek-V2 paper: MLA cache per token

GPU and system specifications

• NVIDIA: L40S specs
• NVIDIA: RTX PRO 6000 specs
• NVIDIA: H100 NVL specs
• NVIDIA: H200 specs
• NVIDIA: DGX B200 specs
• AMD: MI300X specs
• AMD: MI325X specs
• AMD: MI355X specs
• NVIDIA: DGX Spark specs
• Apple: Mac Studio specs
• AMD: Ryzen AI Halo specs

Method, formats and runtimes

• NVIDIA: Mastering LLM Techniques, Inference Optimization (KV cache size)
• llama.cpp quantize README: measured bits per weight
• llama.cpp server README: --swa-full and KV cache types
• Microscaling Data Formats for Deep Learning (MXFP4)
• NVIDIA: Introducing NVFP4
• vLLM engine arguments: --gpu-memory-utilization, --kv-cache-dtype
• vLLM hybrid KV cache manager
• Transformers: Llama 4 layer types
• NVIDIA DGX Spark clustering guide