"How much VRAM do I need" is one of the most common questions from people who want to run AI models on their own machine. You don't need a benchmark site to answer it. You need one formula and a bit of headroom.
Step 1: weights
A model's weights take up roughly:
GB ≈ parameters (in billions) × bits per weight ÷ 8
| Model size | 16-bit | 8-bit | 4-bit |
|---|---|---|---|
| 8B | 16 GB | 8 GB | 4 GB |
| 14B | 28 GB | 14 GB | 7 GB |
| 32B | 64 GB | 32 GB | 16 GB |
| 70B | 140 GB | 70 GB | 35 GB |
| 120B | 240 GB | 120 GB | 60 GB |
Real quantized files usually come out a little bigger than this, because they also store scaling information. Check the actual file size on the model page.
Step 2: context (the KV cache)
While the model works through your prompt, it keeps a KV cache for every token in the context. That cache lives in VRAM too, and it grows with context length.
The formula:
KV cache bytes ≈ 2 × layers × KV heads × head size × bytes per value × tokens
You'll find layers, KV heads and head size in the model's config.json. As a worked example, take a model shape with 32 layers, 8 KV heads and a head size of 128, storing the cache at 16-bit (2 bytes):
- Per token: 2 × 32 × 8 × 128 × 2 = 131,072 bytes (128 KB)
- 32k-token context (32,768 tokens): 4 GiB, about 4.3 GB
- 128k-token context (131,072 tokens): 16 GiB, about 17 GB
So a model that fits nicely at short context can run out of memory when you paste in a whole codebase. Newer models use tricks to shrink this cache; see how DeepSeek V4.1 Flash tackles it.
Step 3: headroom
The runtime, the desktop and other apps also use VRAM. As a rule of thumb, don't plan to fill more than about 85–90% of the card.
Total ≈ weights + KV cache + headroom.
What fits on RTX 50 cards
VRAM and starting prices are from NVIDIA's RTX 50 page as of Oct 22, 2026. "Fits" means weights at 4-bit plus some context, by the arithmetic above.
| Card | VRAM | Memory bandwidth | Comfortable at 4-bit |
|---|---|---|---|
| RTX 5050 (from $249) | 8GB GDDR6 | 320 GB/s | ~8B, short context |
| RTX 5060 (from $299) | 8GB GDDR7 | 448 GB/s | ~8B, short context |
| RTX 5060 Ti 8GB / 16GB (from $379) | 8 or 16GB | 448 GB/s | ~8B (8GB) or ~14B (16GB) |
| RTX 5070 | 12GB | 672 GB/s | ~8B with longer context, ~14B tight |
| RTX 5070 Ti (from $749) | 16GB | 896 GB/s | ~14B with room for context |
| RTX 5080 | 16GB | 960 GB/s | ~14B with room for context |
| RTX 5090 | 32GB | 1,792 GB/s | ~32B (16GB weights) with long context |
Two things to notice:
- Two 16GB cards can feel very different. Token speed depends heavily on memory bandwidth, and the 5060 Ti 16GB has about half the bandwidth of the 5080.
- 70B is a two-GPU (or rental) model. At 4-bit it needs ~35GB for weights alone.
Street prices have moved with the memory crunch, so check current listings. Our explainer on RAM prices covers why.
Buy or rent?
If you only need a big model for a weekend project, renting may be the better deal. On Oct 22, 2026, Runpod listed an RTX 5090 at $0.69/hour (Community Cloud) or $1.19/hour (Secure Cloud). At those rates, 40 hours a month for four months costs about $110–$190. For bigger models, see what it costs to rent an H100, B200 or B300.
To run these numbers for your own model and card, use our local AI hardware sizing sheet.
Running models locally pairs nicely with keeping your data on your own machine. More on that in own your code and data.
