Context windows explained: tokens, 1M-token models and what actually fits

"Context window" is a steady AI topic on X, with about 4,600 posts in our sample and a steady ~600 a day. Every model page shows a number, like 200K or 1M. Here's what that number means for your app or agent session.

What a context window is

Anthropic's docs put it simply: the context window is "all the text a language model can reference when generating a response, including the response itself". It isn't what the model learned in training. It's a "working memory" for one request.

It's measured in tokens, the chunks of text a model reads (often pieces of words). Different models split text differently. Anthropic notes that Claude 4.7 and later models use a newer tokenizer that produces about 30% more tokens for the same text than earlier models. A count from one model doesn't carry over to another.

What uses up the window

Per Anthropic's docs, all of this counts:

• the system prompt
• every message in the conversation, including tool results, images and documents
• your tool definitions (connected MCP servers add these)
• the model's output, including extended thinking

Chats grow every turn: each new request carries the full history so far. That's why a long agent session slows down and gets expensive, even if your latest message is one line.

Typical sizes right now

Two details:

• Gemini 4 Argon shares one budget between input and output, so a huge input leaves less room for the reply.
• Locally, the window you can actually use depends on your memory. Longer context needs more VRAM for the KV cache. See how much VRAM you need.

Why 1M tokens isn't a free pass

1. Context rot. Anthropic: "As token count grows, accuracy and recall degrade." A model can technically fit your whole repo and still miss the one file that matters. Curating what's in context is "just as important as how much space is available."
2. Cost. You pay for every input token, on every turn. A 500K-token conversation re-sends 500K tokens each time.
3. Limits beyond tokens. Claude accepts up to 600 images or PDF pages per request (100 on 200K-window models), and you can hit request-size limits before the token limit.

When you run out

On the Claude API:

• If the input alone is too long, you get a 400 error ("prompt is too long").
• On Claude 4.5 and newer, if input plus max_tokens goes over, the request still runs. If generation reaches the limit, it stops with stop_reason: "model_context_window_exceeded".

In chat apps, the app may quietly drop or summarize old turns. Anthropic notes chat interfaces can manage context on a "first in, first out" basis, which is why a long chat can "forget" early instructions.

How to manage context

• Count before you send. Anthropic's token counting endpoint is free to use, with separate rate limits, and returns an estimate:

bash curl https://api.anthropic.com/v1/messages/count_tokens \ -H "x-api-key: $ANTHROPIC_API_KEY" \ -H "content-type: application/json" \ -H "anthropic-version: 2023-06-01" \ -d '{"model": "claude-opus-5-5", "messages": [{"role": "user", "content": "Hello, Claude"}]}'

• Cache the stable part. Prompt caching makes a repeated prefix much cheaper. It still takes up the window, though: caching changes the price, not the space.
• Compact long sessions. Anthropic offers server-side compaction (beta, Claude 4.6 and later) that summarizes earlier turns, and context editing to clear old tool results.
• Trim tools. Fewer connected tools means fewer tool definitions in every request.
• Start fresh. For coding agents, a new session with a good AGENTS.md often beats a 300-turn session. Subagents also help: each one works in its own context.

Bottom line: the size of the window tells you what fits. What you put in it decides how well the model works.