The most upvoted story on Hacker News this week wasn't a new model. It was the software DeepSeek uses to serve one. DeepInfer went up on GitHub on Oct 16, 2026, and the HN thread passed 2,300 points and 880 comments. By Oct 22 the repo had about 19,800 stars.
DeepInfer comes with a new default checkpoint, DeepSeek-V4.1-Flash-1017. That snapshot got its own HN thread too (959 points). Here's what's actually in the release.
What DeepInfer is
DeepSeek calls DeepInfer "the high-performance inference framework that powers DeepSeek's large-scale production services". The open-source release does offline inference, and it supports one model: V4.1 Flash.
From the README:
- Optimized kernels from DeepSeek's other open repos (FlashMLA, DeepGEMM, DeepEP and others)
- Parallelism: attention data parallel and context parallel; expert parallel for the MoE layers
- Speculative decoding (DSpark) with confidence-scheduled verification
- Batch-invariant inference and deterministic sampling, so the same input gives the same output (NVIDIA only)
- Structured outputs: regex, JSON schema or EBNF grammar
What's not included: a KV cache system, prefill/decode disaggregation, and load balancing across replicas. DeepSeek lists all three as needed for a complete service. So it's an engine, not a drop-in API server.
It was originally built mostly on vLLM, with ideas from SGLang and TensorRT-LLM, and it's Apache 2.0.
The hardware
DeepInfer supports NVIDIA SM100 GPUs (data-center Blackwell) and Huawei Ascend 950 NPUs. It needs Linux, Python 3.12, GCC 13+ and CUDA 13 (or CANN 9.2 on Ascend). DeepSeek's published throughput charts were measured on 4 GPUs for prefill and 64 for decode.
If you want to rent that kind of hardware, our GPU rental price guide has current hourly rates.
What the model card says
The numbers below are from the V4.1 Flash model card. The benchmarks are DeepSeek's own.
| Spec | V4.1 Flash |
|---|---|
| Type | Multimodal mixture-of-experts (images and text in, text out) |
| Backbone parameters | 552B |
| Active parameters per token | 8B (prefill) / 16B (decode) |
| Context | Up to 1M tokens |
| Experts | 1 shared + 384 routed per MoE layer, 6 routed active |
| Pre-training | 45T tokens, from scratch |
| Reasoning effort | Adjustable from 1 to 100 |
| License | MIT (code and weights) |
The headline is KV cache size. Long contexts get expensive because the model keeps a cache for every token, and that cache eats memory. DeepSeek says V4.1 Flash cuts the global KV cache to 890 bytes per token, about a quarter of V4 Flash. It does this with a new attention scheme (CSA2) and an FP4 cache. The persistent cache footprint drops to about an eighth. For agents that read huge codebases or long tool logs, that's the cost that matters.
A few vendor-reported scores at maximum effort, against DeepSeek's own V4 Flash:
| Benchmark | V4 Flash | V4.1 Flash |
|---|---|---|
| Terminal-Bench 2.1 | 82.7 | 90.6 |
| DeepSWE v1.1 | 54.4 | 74.2 |
| GPQA Diamond | 89.9 | 90.9 |
The card also compares against closed models. On some agent benchmarks V4.1 Flash leads; on harder ones, like Terminal-Bench 3.0 and 4.0, it trails the top closed models by a wide margin. Read the whole table, not the cherry-picks.
Can you run it at home?
Short answer: no, not on one consumer card.
Some quick arithmetic: at 4 bits per weight, 552B parameters is about 276 GB just for weights, before any cache. The biggest current GeForce card, the RTX 5090, has 32GB. Our VRAM guide walks through the math for models you can run locally.
For most builders the practical way to try V4.1 Flash is a hosted API. Hugging Face lists several inference providers for it, and the model card notes that the release has no Jinja chat template. DeepSeek ships a Python reference encoder and a Rust library, deepseek-recipe, instead. If you're wiring it into your own tooling, read that part of the card first.
Why it matters
Open weights already let you run a model. Open-sourcing the production engine shows you how the people who built it run it cheaply, and that knowledge tends to flow into vLLM, SGLang and the hosted providers within weeks.
