China's open-weight AI models are now just 4 months behind frontier US offerings, Mozilla report claims
Even at Deepseek's own pricing (remember, this is an open-weight model you can run anywhere), the price at off-peak times per million tokens is just $0.15 for input and $0.6 for output. On cache hits, the input token cost drops to a mere $0.003. It's worth keeping in mind that third-party providers have undercut Deepseek API in the past, so those figures could go lower if you shop around inference providers. Deepseek also offers search functionality at $0.75 per 1,000 queries, an amount that undercuts competitors , with the closest one being Parallel at $1.
DeepSWE isn't the only benchmark around, either. Artificial Analysis ranks DSF-1017 (max) as only #2 on its intelligence index within the large open-weight class, with a score of 45. For reference, the most seemingly intelligent model, Claude Opus 5.5 (max with fallback), rings in at 58. It's one of the fastest models in its category, too, and its cost-per-task is edged out only by Xiaomi's MiMo V2.6-Pro.
This all sounds too good to be true, but everyone knows the adage of "fast, cheap, and reliable, pick two." So what's the catch? In one word: hallucination. Just because a model is smart doesn't mean it's reliable. Broadly speaking, users prefer models that don't answer when they lack supporting facts, and DSF-1017 doesn't seem very good at knowing what it doesn't know.
Artificial Analysis' Omniscience benchmark is quite clear on this matter, ranking the model among the lowest. It answered questions correctly 43% of the time, which is among the best for open-weight models, but inferior to the proprietary competition. That would be perfectly acceptable if it skipped answers without a foundation, but it hallucinated 67% of non-answers.
The benchmark's main index grants points for good answers and deducts them for wrong ones, and non-answers don't count. DSF-1017 gets 5 points, compared to Kimi K3-max's 20, with the frontier models coming in at 42+. Ultimately, this means DFS-1017 is by no means incompetent — much the opposite —but one should be careful about which tasks to assign to the model.
Tasks with verifiable outputs like programming, OCR with review, and format conversions will be fine, as the agent can run verification steps against its work and try again. However, open-ended research, looking up facts, and interpreting or drafting highly precise documents are tasks better suited to a frontier model — or, you know, a human — as hallucinations likely won't be properly reined in and results won't be very trustworthy.
"Fast and loose" seems to be a forming trend for highly competent open-weight models, too. Several months ago, when Kimi K3 was released , everyone raved about how effective and cheap it was, and how it proposed to offer frontier-level intelligence for a fraction of the price. My personal experience with Kimi was short-lived, as even short but precise fact look-ups came up with hallucinated information, and Artificial Analysis' data seems to back this observation up.
Nevertheless, open-weight models remain important both to make AI agents more accessible in absolute dollar terms and potentially as a tool to drive down the cost of intelligence when facing down potential price rises from frontier labs.
Bruno Ferreira Social Links Navigation Contributor Bruno Ferreira is a contributing writer for Tom's Hardware. He has decades of experience with PC hardware and assorted sundries, alongside a career as a developer. He's obsessed with detail and has a tendency to ramble on the topics he loves. When not doing that, he's usually playing games, or at live music shows and festivals.
Key considerations
- Investor positioning can change fast
- Volatility remains possible near catalysts
- Macro rates and liquidity can dominate flows
Reference reading
- https://www.tomshardware.com/tech-industry/artificial-intelligence/SPONSORED_LINK_URL
- https://www.tomshardware.com/tech-industry/artificial-intelligence/deepseeks-latest-ai-model-is-smarter-than-ever-but-its-low-running-costs-come-at-a-price-open-weight-model-is-fit-for-agentic-workloads-but-is-more-prone-to-hallucination-than-proprietary-competitors#main
- https://www.tomshardware.com/my-account
- EA is closing BioWare, the studio behind Mass Effect and Dragon Age, after more than 30 years
- Sony Japan marks last standalone 4K Blu-ray player as ‘Discontinued’
- AMD Ryzen 5 5500F review: Vermeer rides again for budget gaming PCs
- Samsung and SK Hynix are upgrading Chinese factories to produce advanced RAM chips as AI data center competition ramps up
- NVIDIA cuDSS Now Generally Available to Accelerate Engineering and Scientific Simulations
Informational only. No financial advice. Do your own research.