Tool 001 · LLM inference

How much VRAM does this model need?

Estimate weights and KV cache separately, add runtime overhead and a safety margin, then compare the result with a GPU memory class.

Open calculator All infrastructure tools

StatusFree to use · browser-onlyEvery Run Any AI resource declares what is real today.

Current estimate

Inference memory plan

Change any assumption and the result updates immediately. Nothing is sent to a server.

Model architectureRepresentative architectureExamples fill architecture fields; use the exact model config for a deployment decision.

ParametersbillionWeight precisionbitsTransformer layersHidden sizeAttention headsKV headsEqual to attention heads for MHA; lower for GQA/MQA.

Serving assumptions

Context lengthtokensConcurrent sequencesKV cache precisionbitsRuntime overhead% of weightsSafety margin%Memory per GPUGiB

Formula usedweights = parameters × bits ÷ 8KV = 2 × layers × KV heads × head size × bytes × context × sequences

Values are converted to GiB, then the selected runtime overhead and safety margin are added. MoE models need the total parameter count resident in memory, not only active parameters.

Read the result

Weights are only the first line.

Quantized weights often dominate a small-batch deployment, but KV cache grows with layers, KV heads, head dimension, context length, and concurrent sequences. Long-context or high-concurrency serving can make the cache a first-class capacity item.

The calculator models decoder-only inference. It does not estimate optimizer state or gradients for training.

Why use GiB instead of GB?

GPU memory is commonly discussed in binary units. The calculator converts bytes using 1 GiB = 1,073,741,824 bytes.

Does 4-bit quantization mean exactly 0.5 bytes per parameter?

That is the planning assumption for weight data. Real formats can add scales, zero points, metadata, padding, and unquantized tensors, which is why overhead and validation matter.

What about mixture-of-experts models?

Use the total parameters that must remain resident, not only the active parameters per token. Runtime-specific expert offload changes the answer.

Can one model span several GPUs?

Yes, but a minimum aggregate-memory count does not guarantee the split will work. Interconnect, tensor or pipeline parallelism, per-device buffers, and uneven layer placement all matter.