Mistral Large 4 self-hosting costs versus the API

The tempting comparison is a cheap-looking hourly GPU price against a large monthly API bill. It leaves out the important part: how much useful work the rented machine finishes, and how long it sits waiting for requests. A server that fits the weights is not necessarily one that meets your latency target. For current ChatGPT, Claude and Gemini list prices, see our AI models explained guide.

What the released model actually needs

Mistral's model card describes about one trillion total parameters, with 49 billion active per token, or 52 billion including embeddings and output layers. The two active-parameter figures count different components; they are not competing estimates.

Mixture-of-experts routing reduces the computation applied to each token. It does not mean only the active experts need storage. At a simplified one byte per parameter, one trillion parameters implies about one terabyte for weights alone; at half a byte, about half a terabyte. Those are our theoretical calculations, not measured checkpoint sizes or a deployment specification. Scales, higher-precision components, the vision encoder, runtime buffers and the key-value cache add overhead.

The official NVFP4 checkpoint mixes NVFP4 experts with FP8 linear layers and targets NVIDIA Blackwell hardware. Mistral publishes vLLM and SGLang serving recipes. Its SGLang NVFP4 example uses tensor parallelism across four GPUs; the vLLM example uses eight. Neither makes four arbitrary GPUs interchangeable with eight H200s.

Hugging Face's Oct 29 deployment article describes an FP8 endpoint recipe on eight H200 GPUs. That recipe serves up to 131K tokens of context, not the model's entire advertised one-million-token window. A model's maximum supported context and the context configured on a particular server are different things. Our VRAM sizing guide explains why concurrency and context need memory too.

Three bills to compare

This is an operating-cost comparison, not a claim that every route produces the same speed or quality. Keep the model checkpoint, quantization, reasoning effort and task set fixed where possible.

Calculate the API baseline first

Mistral's release page, checked Nov 5, lists $0.70 per million input tokens, $0.07 per million cached input tokens and $2.10 per million output tokens. The following is an illustrative calculation using those rates, not a customer bill:

• Ten million uncached input tokens: $7.00.
• Two million output tokens: $4.20.
• Total: $11.20.

If all ten million input tokens instead qualify for the listed cached-input rate, the input portion becomes $0.70 and the total becomes $4.90. That is a separate hypothetical, not a promise that every prompt will hit a cache. See how prompt caching changes a bill.

Count the tokens the provider actually bills. The model card notes that its completion-token limit includes reasoning as well as visible output. An answer's visible length alone is therefore not a reliable workload budget.

Price the rental without pretending it is a quote

Runpod's pricing page, updated Nov 4 and checked Nov 5, lists H200 Secure Cloud Pods at $5.29 per GPU-hour. Eight times that rate is $42.32 an hour, or $30,470.40 for an assumed 720-hour month. Those are our multiplications of a listed per-GPU rate, before storage and any other charges.

That calculation is a budgeting reference, not a verified offer for a compatible eight-H200 node. Eight separately available GPUs do not establish the required topology, bandwidth, region or inventory. The same page lists H200 SXM Clusters separately at $6.35 per GPU-hour, illustrating why a Pod price should not silently become a cluster quote. Obtain the rate for the actual deployment being tested.

Against the $11.20 API example, one hour at the illustrative $42.32 rental rate costs as much as about 3.78 copies of that token bundle. That is a cost-equivalence calculation only. We have not measured whether the node can finish those bundles in an hour at an acceptable latency, or whether demand would keep it busy.

Our GPU rental guide covers the wider market. For this decision, use the node you can actually provision, not the lowest isolated accelerator price.

Ownership needs an installed-system budget

An owned machine removes the rental invoice, not the costs. A workable comparison allocates the installed purchase price across the months you expect to use it, then adds power, cooling, connectivity, maintenance, staff and any financing. Include backup capacity if your application cannot stop during a failure.

We are not assigning a dollar price to owned hardware here: no matched, installed-system quote was verified for this comparison. A number inferred from a single accelerator listing would hide the cost of the rest of the system.

The key denominator is useful output. Compare cost per successfully completed task under a latency target, rather than advertised tokens per second alone. Benchmark short and long prompts, realistic concurrency and your chosen reasoning effort. Record failed requests and retries as costs, not free throughput.

Check the licence before the break-even spreadsheet

The Mistral Ecosystem Open-Weight License permits use, modification and distribution subject to its conditions. If the licensee or an affiliate operates a Model-as-a-Service business, a separate commercial licence is required before commercial use, subject to the licence's exceptions for internal use and use through Mistral's official products.

Its definition excludes end-user products where model capabilities are solely embedded within specific features or harnesses, and merely relaying requests to models hosted by others. Do not assume every application is Model-as-a-Service, or that downloadable weights mean unrestricted hosting rights. No commercial licence price is assumed in the calculations above.

For a deployment test, keep the service private and enforce authentication before exposing it beyond the test environment. The vLLM server guide covers the serving interface. A useful final comparison has three rows: measured API cost, measured rental cost at the same quality and latency, and a fully quoted ownership budget. Until those rows describe equivalent work, there is no defensible universal break-even point.