Freely downloadable language models now appear in the same comparison as the best proprietary systems, but they do not sit at the top of it. On Artificial Analysis's current index the leading open-weight model scores roughly half what the leading closed model does. The gap has narrowed. It has also stopped being the only question worth asking.

What the numbers say

Artificial Analysis runs a public index that now incorporates nine evaluations, including agentic software work on Terminal-Bench 4.0, long-document reasoning on GDP.pdf, and hard science questions. The current version ranks 763 models.

At the top, the leaders are proprietary: Claude Fable 5.2 (max) scores 65 and GPT-6.2 Astra (max) scores 62.

The strongest open-weight entry is DeepSeek V4.1 Flash 1017 (max) at 33, a 552-billion-parameter model with 16 billion parameters active at inference and a one-million-token context window. Its sibling DeepSeek V4 Pro 0813 (max) scores 23, and Qwen3.8 27B (xhigh) scores 20. Artificial Analysis lists the first at US$0.18 per million tokens.

So: a 33 against a 65, at a fraction of the price. Whether that is a triumph or a rounding error depends entirely on what you are asking the model to do.

Why 33 is not half a model

An index is an average, and averages hide shape. A model can be within a point or two of the leader on the bulk of everyday prompts and much further behind on the awkward ones, and the average will describe neither case.

The evaluations in this index lean towards the awkward ones. Terminal-Bench asks a model to complete multi-step work in a terminal. GDP.pdf asks it to reason across long documents. HLE-Diamond and CritPt are deliberately hard. These are closer to the tasks a business wants handled unattended than to drafting a reply to an email.

That matters commercially, because the value of a model is not the average answer. It is whether you can leave it running without a person checking every third output. The tail is where the money is, and the tail is where open weights are still furthest behind.

The price argument, done properly

Cheap tokens are not the same as a cheap product. The useful figure is cost per completed task, which Artificial Analysis also publishes, because a model that needs three attempts and a human review is not cheaper than one that works first time.

There is a second cost that never appears on a pricing page: running the model yourself. Weights are free to download and expensive to operate. You need hardware, a serving stack, security patches, and an evaluation set good enough to notice when a new release quietly gets worse at your particular job. Teams that skip the last item are not self-hosting, they are hoping.

That operational bill is why plenty of organisations will keep paying for access even when a free model scores close enough on their own tests.

Licensing is the quiet variable

"Open weights" is not one thing. Artificial Analysis separates models released under a permissive open licence from those marked "open weights, commercial use restricted", which covers releases you can download and inspect but cannot build an unrestricted business on.

Several of the strongest downloads carry terms that restrict commercial use above a revenue threshold or exclude certain kinds of product. That turns a technical decision into a legal one, and it is the first thing to read before committing engineering time.

What changed to get here

Two years ago an open model strong enough to be useful needed a workstation and still felt slow. The arrival of reasoning models released with downloadable weights, beginning with DeepSeek R1 in January 2025, changed the expectation: a model near the frontier could be inspected, fine-tuned and run on your own metal.

Since then several labs have settled into a rhythm of shipping weights alongside hosted access, and the open-weight entries now appear in the same top-tier comparison as the proprietary leaders rather than in a separate league below them.

What to watch

Three things, in order of usefulness to a buyer:

  • Cost per completed task, not cost per token. It is the only figure that reflects retries and failures.
  • Performance on the evaluations closest to your work. A general index is a starting point, not a verdict.
  • Licence terms and release cadence. A model you cannot legally deploy, or that stops being maintained, is not a bargain.

The short version

Open weights have moved from "good enough for hobbyists" to "good enough to be on the same chart as the leaders", at a price that changes the economics of high-volume work. They have not closed the gap at the top, and the tasks where the gap is widest are the ones that decide whether a product can run without supervision. That is a healthier market than the one we had, and a more complicated one to buy in.