How much VRAM do you need to run an LLM locally? The simple math
A plain-English formula for working out how much GPU memory a local AI model needs, with worked examples for 8GB, 12GB, 16GB and 32GB cards.
Local AI hardware sizing sheet: VRAM calculator for running LLMs on RTX 50 cards
A spreadsheet and a Python script that work out how much VRAM a local model needs (weights plus KV cache) and which RTX 50 cards fit it. Card specs from NVIDIA, checked Oct 22, 2026.
Want to know if a model will run on your card before you download 20GB? This sheet does the arithmetic from How much VRAM do you need to run an LLM locally? for you.
Preview: vram_calc.py (core math)
GIB = 1024 ** 3
def weights_bytes(params_b, bits):
return params_b * 1e9 * bits / 8
def kv_bytes(layers, kv_heads, head_dim, tokens, kv_bits=16):
return 2 * layers * kv_heads * head_dim * (kv_bits / 8) * tokens
def fits(need_bytes, vram_gib, headroom):
return need_bytes <= vram_gib * GIB * headroom
Example: an 8B model at 4-bit with a 32k context, using the article's worked shape (32 layers, 8 KV heads, head size 128), needs about 3.7 GiB for weights and 4 GiB for the KV cache. That's too big for an 8GB card at 85% headroom, but fits a 12GB card.
What's inside
local-llm-vram-sizing.xlsx: the calculator, with an RTX 50 table that marks each card "fits" or "too big"vram_calc.py: the same math on the command line, with--configto read a model'sconfig.jsonrtx-50-vram.csv: VRAM, memory type, bandwidth and starting prices from NVIDIA, checked Oct 22, 2026model-weights-gb.csv: weight sizes for 8B to 120B models at 16, 8 and 4 bitstests/: unit tests for the formulas
How to use it
- Open the spreadsheet (or import it into Google Sheets).
- Fill in the yellow cells: parameters, bits, layers, KV heads, head size and context.
- If you know the real file size, enter it. It overrides the formula.
- Read the "Fits?" column.
Who it's for
Anyone choosing a GPU, or a model for the GPU they already have.
Related
FAQ
Where do I find layers, KV heads and head size?
In the model's config.json on its model page: num_hidden_layers, num_key_value_heads and head_dim. If head_dim is missing, it's hidden_size divided by num_attention_heads.
Why does the sheet say GiB?
GPU memory is counted in binary units (1 GiB = 1,073,741,824 bytes), so the sheet converts everything to GiB before comparing it with a card.
Does it work for Apple Silicon or AMD cards?
The math does. Put your memory size in the 'Your card' row (or use --vram) instead of reading the RTX 50 list. On a Mac, leave room for the system, since memory is shared.
Why does a model that 'fits' still run out of memory?
Real quantized files are often a bit bigger than the formula, and runtimes add their own buffers. Use the real file size and keep the headroom at 85–90%.
Keep reading
Keep reading
DeepSeek open-sources DeepInfer: what V4.1 Flash is, and can you run it yourself?
DeepSeek released DeepInfer, the inference engine behind its own service, plus a new V4.1 Flash checkpoint (1017). What the model card actually says, and the hardware DeepInfer needs.
Newsletter
The Codema publication. Consent is required. You are not subscribed until you confirm from the email, and nothing is stored if email delivery is not connected.