Rate limits and 429 errors: keep your AI app working under load

You ship a feature, traffic arrives, and requests to your model provider start failing with HTTP 429. The provider isn't down. You've hit one of its limits. Here's how those limits work at the three biggest APIs, how to tell a 429 you can retry from one you can't, and the patterns that keep an app working when it's busy.

What gets limited

Each provider measures several things at once, and going over any one of them triggers a 429:

• Requests per minute (RPM). Twenty tiny requests can use up an RPM limit of 20 even if you sent very few tokens.
• Tokens per minute (TPM). Anthropic splits this into input tokens (ITPM) and output tokens (OTPM). Google's TPM counts input.
• Daily limits. OpenAI lists requests and tokens per day (RPD, TPD) for some models. Google's requests per day reset at midnight Pacific time.
• Spend limits. Monthly caps at Anthropic and OpenAI, and a rolling 10-minute spend limit at Google for paid tiers.

Limits are shared more widely than many people expect. Anthropic sets them per organization, OpenAI per organization and project, and Google per project, not per API key. So a second key won't get you more capacity.

Limits also apply over shorter windows. Anthropic says a 60 RPM limit might be enforced as one request per second, and it uses a token bucket that refills continuously rather than resetting each minute. Bursts can fail even when your per-minute average looks fine.

The tiers, as of Oct 28, 2026

All three providers move you up tiers automatically as you spend. Your own numbers are in each console.

Sources: Anthropic, OpenAI and Google rate-limit docs. Google's spend-based limit is $10, $50 and $200 per 10 minutes for Tiers 1 to 3. For what each model costs per token, see our LLM API pricing comparison.

Read the 429 before you retry

A 429 can mean "slow down for a few seconds" or "stop until someone changes a setting." Check the body and headers first.

• Anthropic, temporary limit: error type rate_limit_error, usually with a retry-after header in seconds.
• Anthropic, monthly spend cap: the same rate_limit_error type, but no retry-after, and error.details.error_code is enforced_spend_limit_reached. Anthropic says retries, including the SDK's automatic ones, keep failing until access resumes on the first of the next month or you move up a tier.
• Anthropic, your own lower spend limit: HTTP 400 with invalid_request_error, not a 429.
• OpenAI, ramping too fast: 429 with rate_limit_error and code slow_down. This can happen even within your RPM and TPM limits. OpenAI's rule of thumb is that once you're at 1M TPM, you should grow traffic by no more than 50% every 15 minutes.
• OpenAI, model overloaded: 503 with server_is_overloaded. Retry it, with longer gaps if it persists.
• OpenAI, hard spend limit: a 429 you shouldn't retry. OpenAI says Retry-After doesn't mean quota or billing errors can be fixed by retrying.
• Google: 429 RESOURCE_EXHAUSTED, including when you hit the spend-based limit. Google's advice is to wait and retry, reduce expensive requests, or request an increase.

The headers worth logging

Anthropic returns anthropic-ratelimit-requests-remaining, anthropic-ratelimit-input-tokens-remaining, anthropic-ratelimit-output-tokens-remaining and matching -limit and -reset headers, with reset times in RFC 3339 format. OpenAI returns x-ratelimit-remaining-requests, x-ratelimit-remaining-tokens and x-ratelimit-reset-requests and -tokens, with resets as durations such as 6m0s. Logging the remaining values tells you which limit you're closest to before you hit it. Google's rate-limit page doesn't document equivalent headers. It points you to AI Studio instead.

Retry with backoff and jitter

When a retry makes sense:

1. If retry-after is present, wait at least that many seconds. OpenAI says to treat it as a minimum and add a small random delay.
2. If it's missing, use exponential backoff: wait about a second, then double the wait after each failure, with random jitter so many clients don't retry at the same moment.
3. Cap the number of attempts and the total time. If the server asks for a longer wait than you allow, give up or defer the job instead of retrying early.
4. Don't stack retry loops. The official OpenAI and Anthropic SDKs already retry some errors, so either turn SDK retries off or count them in your limits.
5. Don't automatically replay a streaming request after output has started arriving.

A minimal version in Python, without any SDK:

RETRYABLE = {429, 503}

def call_with_retries(send, max_attempts=6, max_total_s=90):
    import random, time
    start = time.monotonic()
    delay = 1.0
    for attempt in range(1, max_attempts + 1):
        resp = send()
        if resp.status_code not in RETRYABLE:
            return resp
        if is_spend_cap(resp) or attempt == max_attempts:
            return resp  # needs a human, or we're out of attempts
        wait = retry_after_seconds(resp)
        if wait is None:
            wait = delay + random.uniform(0, delay)
            delay = min(delay * 2, 30)
        else:
            wait += random.uniform(0, 1)
        if time.monotonic() - start + wait > max_total_s:
            return resp  # defer the job instead of retrying early
        time.sleep(wait)

retry_after_seconds reads the retry-after header and returns None if it's missing or invalid. is_spend_cap checks for Anthropic's enforced_spend_limit_reached code, or your own billing-error rules for other providers.

Stop hitting the limit in the first place

• Queue and pace. Put model calls behind a queue with a concurrency limit, and use the remaining-requests and remaining-tokens headers to slow down before you hit zero.
• Cache repeated context. At Anthropic, tokens read from the cache don't count toward ITPM on most Claude models. Its docs give the example of an 80% cache hit rate turning a 2M ITPM limit into about 10M total input tokens a minute. How caching works: prompt caching explained.
• Send less. Trim conversation history and retrieved documents. A huge context eats your TPM budget on every call: see context windows explained.
• Set output size sensibly. OpenAI estimates usage from the larger of max_tokens and your prompt's estimated size, so an oversized max_tokens uses up TPM. Anthropic says max_tokens doesn't count toward OTPM.
• Use batch APIs for work that can wait. Anthropic, OpenAI and Google all have batch limits that are separate from the live ones.
• Ramp up gradually. Anthropic warns about acceleration limits on sharp increases, and OpenAI has its slow_down error.
• Have a fallback. For user-facing features, route to a second model or provider when the first is limited, and tell the user if a reply will be slow.

Coding agents have their own limits

Subscription tools like Claude Code, Codex and Cursor have usage limits that are separate from API rate limits, and they're set by each plan. If you run Claude Code through the API, Anthropic notes that requests over the Claude Code workspace's limit can return a 429 with a retry-after header. Running agents unattended? Give them tight permissions too: see how to run coding agents safely.