Quick Answer
Open-source and open-weight AI models have caught up for most everyday company work. They are not better at everything. On the hardest, longest, most open-ended tasks, the best closed models still lead. But for classification, extraction, summarising, drafting, coding help, search summaries, and structured work, the difference is now small, the cost difference is large, and the control difference matters more than either.
Mozilla’s State of Open Source AI report, published September 15, 2026, puts the gap at about 4.4 months. That is the time it takes the best open models to reach the level the closed frontier reached earlier, measured by the length of task a model can complete reliably. Most companies should now start with open models as the default, and pay the frontier premium only where the work earns it.
That is the short version. The rest of this article explains the numbers, what three companies actually did, and how to run a small test of your own before you commit to anything.
The gap, in plain numbers
Two years ago, this was not a close call. If you wanted the best reasoning, the best coding help, or the most reliable multi-step work, you paid for a proprietary API. Open models were good for demos and side projects. They were not the safe choice for production.
The Stanford AI Index documented a 17.5 percentage point gap between the best US and Chinese models on MMLU at the end of 2023. By early 2026, that gap had effectively reached zero on that test. That does not mean every open model beats every closed model on every test. It means the old assumption, that only closed labs can build frontier quality, no longer holds.
Here is how the gap looks now, on tests that resemble real work more closely than a quiz:
On SWE-bench Verified, a set of 500 real GitHub issues from production Python projects, the best open-weight model, Moonshot AI’s Kimi K2.5, scored 76.8% in February 2026. The best proprietary model, Claude Opus 4.5, scored 80.9%. Four points. Two years earlier, the open field was more than twenty points behind.
On maths, DeepSeek V3.2 scored 96.0% on AIME-2025, ahead of GPT-5 High at 94.6%, at roughly one tenth of the API cost, according to reporting on the release. Treat vendor benchmark numbers with care. They are useful as a direction of travel, not as a promise for your workload.
Mozilla’s measure is more useful for a manager than a benchmark score. It uses work from METR, a nonprofit that evaluates models by the length of expert task they can complete at least half the time. On that measure, the best closed model can handle a task that would take a human expert about 12 hours. The best open model can handle about a 7 hour task. The open model reaches the 12 hour level about four months later. Open capability is also improving faster on this measure, doubling every 3.9 months against 5.5 months for closed models.
Put simply: for work under about eight hours of human-equivalent complexity, open models are competitive. Past that, neither category is reliable yet, and the closed models hold a clearer lead in the 8 to 12 hour range.
That framing changes the buying question. It is no longer “is open good enough?” It is “is this particular job long and messy enough to justify paying five times as much for a four month head start?”
What actually changed
Three things changed at once. Any one of them alone would have mattered. Together, they changed the default.
1. Capability caught up, in several places at once.
It was not one lab. DeepSeek, Qwen from Alibaba, Kimi from Moonshot, GLM from Z.ai, and Mistral all reached frontier quality on different tests around the same time in late 2025. That matters because it removes single vendor risk. If one family stalls, another is close behind.
2. Cost collapsed.
Inference costs have dropped by more than 99% in two years, driven in large part by open competition and more efficient mixture-of-experts designs. Closed models still cost, on average, several times as much as open alternatives per million tokens in the comparisons I read for this article, and DeepSeek’s API pricing was reported at about 95% cheaper than OpenAI’s o1 for comparable reasoning work. Cost per token is not the whole story, and I will come back to total cost. But when the price gap is that wide, “just use the frontier for everything” stops being a neutral default. It becomes an expensive habit.
3. Licensing got simpler.
Through 2024, open-weight licensing was a patchwork of custom terms. By mid-2026, the dominant licences are Apache 2.0 for most of the Qwen open-weight line, Gemma, and Mistral, and MIT for DeepSeek. Both are well understood by legal teams and impose minimal friction on commercial use.
There is an important distinction here, and it is worth stating plainly. “Open weight” is not the same as “open source” in the strict sense. You can download the weights, run them, and fine-tune them, but you may not get the training data or the full training pipeline. Meta’s Llama also has its own community licence, with a cap on free commercial use at 700 million monthly active users. And some flagship models, such as Qwen’s largest model, are proprietary and API only, while only the smaller models in the line are open. Check the licence for the exact model and version you plan to use, not just the family name.
None of this makes open models free. It makes them controllable. That is a different benefit, and for many companies, the more durable one.
Three case studies, with the caveats left in
Case studies in AI articles are usually too clean. These three are useful precisely because they are messy, partial, and specific. I have left in what is company-reported and what is not yet independently verified.
1. Ecosia: switching models twice, and halving costs on the second switch
Ecosia is the Berlin search engine that spends its profits on planting trees. In May 2026, it replaced OpenAI with Mistral as its AI provider, hoping for a more European, more sovereign setup. Five months later, in early October 2026, founder and chief executive Christian Kroll told Politico that Ecosia had moved much of its AI to open-weight models, Qwen, GLM, and Kimi, served through Melious, a German platform, on European servers.
Kroll said the switch had roughly halved Ecosia’s AI costs while improving quality and performance, and that Mistral’s models were a year behind the competition and its servers were overloaded. That cost figure is Ecosia’s own claim. Politico published no baseline, benchmark, or invoice data to test it against, and Ecosia has not published which specific model versions route which traffic. Treat “roughly half” as a reported result for Ecosia’s workload, not as a price you will get.
What is verifiable, and more interesting than the headline number, is the structure of the decision. Ecosia did not pick one model and marry it. It moved to a platform that hosts a menu of open models, so it can switch routing without rebuilding the product. It kept data processing inside the EU, on European infrastructure. And it kept OpenAI listed as a fallback in its privacy policy, for when its main provider is down. That is routing, not religion. It is a good model for how a mid-sized company can use open models without pretending the decision is permanent.
There is also a sovereignty wrinkle worth noting. The models Ecosia adopted come from Chinese labs, even though the hosting is European. Kroll has said sensitive subjects can be routed to alternative models, and that the company is working to strip political bias from answers. For a company in a regulated or politically sensitive area, where a model was built and where it runs are two separate questions. Both need an answer.
2. Shopify: a small, specialised model beating the frontier on one job
At PyTorch Conference North America in October 2026, Shopify described a continual learning loop built with PyTorch and vLLM, both open-source projects. The loop captures production failures from its Sidekick GraphQL agent, repairs them with a calibrated judge, and folds the successful repairs into the model weights every day through supervised fine-tuning and reinforcement learning.
The result, as Shopify reported it: the specialised model matched or exceeded frontier-model quality on that specific GraphQL task, while cutting serving costs by 96% for that workload. Shopify estimated serving the same traffic on a frontier model at roughly $27 million a year at average token prices, against closer to $1 million for the fine-tuned model.
Read that carefully. It is a serving cost estimate for one workload, not a total cost of ownership figure. It does not include training, engineering time, or the cost of building and running the learning loop. It also does not prove that a small model beats the frontier in general. It proves something narrower and more useful: for a well-defined, high-volume, repeatable job, a small model trained on your own production failures can beat a large general model, and cost far less to run.
That is exactly the pattern I wrote about in The Quiet Revolution: How Small AI Wins Drive Better ROI Than Massive Rollouts. Small wins, close to the work, with a feedback loop, beat a big rollout that tries to do everything at once. Shopify is that idea, applied at the model level.
3. Nous Research: open tooling turning into enterprise revenue
Nous Research builds Hermes, an open-source agent model and toolset. In October 2026, the company confirmed a $90 million Series B at a $1.5 billion valuation, led by Robot Ventures, with Nvidia, Samsung, Union Square Ventures, Menlo Ventures, and 1789 Capital taking part, bringing total funding to $158 million.
The business signal, as reported by the Wall Street Journal and summarised in trade coverage, is that Hermes had been cloned over 24 million times and was driving about 2.5% of global AI token usage, with annualised revenue near $36 million as of mid-September and a target above $100 million by year end. Treat those revenue figures as reported, not audited.
Why include a startup funding story in a piece about company buyers? Because it answers the support question that rightly worries enterprise teams. Open does not have to mean unsupported and alone. There is now a real market in companies that take open models, add the hosting, evaluation, security, and support layer, and sell that as a product. You can buy the safety net without buying the model itself. That distinction, model versus managed service, is where a lot of 2026 buying decisions will actually be made.
A smaller case, from my own desk
I run open models at home, on my own machine, in ComfyUI. Qwen image models, local language models, workflows I can inspect, change, and rerun without sending a prompt to anyone else’s server. That is a hobby setup, not an enterprise deployment, and I would not pretend otherwise.
But it taught me the same lesson the bigger cases teach, at a scale I could feel. When I control the model, the prompt, and the workflow, I can iterate fast, I can reproduce a result, and I can see exactly where it went wrong. When something fails, I fix the workflow, not my relationship with a vendor. Companies get the same benefit at a larger scale, with more at stake: control over data, over behaviour, and over cost, in exchange for owning more of the engineering.
If you have never run a model locally, try it on one small job before you decide what you think about open models in general. The experience of running it yourself changes the conversation from abstract to practical very quickly.
Five ways companies actually benefit
1. Cost, where volume is high and the job is well defined.
This is the clearest win, and the easiest to measure. High-volume summarising, extraction, classification, drafting, and coding assistance are where open models now match closed ones closely enough that paying the frontier price for every call is hard to justify. Measure cost per successful task, not cost per token. A cheap model that needs three retries is not cheap.
2. Data stays where you put it.
A hosted API sends your prompts to someone else’s servers. A self-hosted open model does not. That distinction matters most in healthcare, finance, legal work, and any company handling personal or commercially sensitive data. Note the nuance: self-hosting a model on infrastructure you control is materially different from using the same model’s hosted chatbot app. The app may store data elsewhere. The weights on your server do not phone home on their own. Check the hosting setup, not just the model name.
3. You can shape the model to your work.
Fine-tuning, and simpler techniques like good retrieval and good prompts, let you teach a smaller model your terminology, your formats, and your edge cases. Shopify’s loop is the advanced version. A simpler version is a model tuned for your support tickets, your product names, and your tone, instead of a general model guessing at all of them. This is also why open models can produce less generic prose when teams iterate on them for brand voice, a point several enterprise teams raised in 2026 coverage.
4. Less lock-in, more negotiating power.
Even if you keep using closed models for some work, having a credible open alternative changes the negotiation. You can route work by cost and quality, switch providers without rewriting everything, and avoid a single price rise or policy change breaking your product. Ecosia’s menu approach is the pattern. Use more than one model, on purpose.
5. Privacy and compliance become engineering choices, not vendor promises.
With an open model, you own the logging, the retention, the access controls, and the audit trail. That is more work, and I will be honest about that below. But it also means you can answer a regulator, a client, or your own security team with evidence from your own systems, instead of a vendor’s assurance page. For companies in the EU, the EU AI Act does not exempt open-source models from core obligations, and general-purpose AI provider requirements have been in force since August 2025, with Commission fines possible from August 2026. Open does not mean unregulated. It means the responsibility sits with you, in a way you can actually discharge.
When to still pay for the frontier
This is the part most open-source enthusiasm skips. Do not switch everything. Pay for the frontier where the work earns it:
- Long, multi-step, agentic work where failure is hard to spot until late. This is where the gap is widest. On SWE-bench Pro, a tougher, standardised coding benchmark, the gap was 18.2 points in one 2026 analysis, against 0.2 points on the older, saturated SWE-bench Verified. On Humanity’s Last Exam, the hardest general benchmark in that analysis, the gap was 9.8 points. Benchmarks move, and vendors report their own numbers optimistically. The direction is consistent: the harder and longer the task, the more the frontier still earns its price.
- Work where a wrong answer is expensive and hard to detect. Legal reasoning, complex financial analysis, and safety-critical drafting belong here until you have strong evaluation and human review in place.
- Work where you need the vendor’s safety, support, and compliance layer, and you do not have the team to build your own yet. That is a legitimate reason to stay closed for now, and a good reason to plan the open pilot in parallel.
- Brand new capabilities, in the first few months after release. The frontier leads, open catches up in about four months. If being first matters more than cost, pay for first. If being right and cheap matters more, wait.
A useful rule from Mozilla’s framing: use open as the default for the majority of work, and pay the premium only for specialised, high-intensity, or long-context scenarios that need it. I would add one word: test. Default does not mean automatic. It means start there, and make the closed model prove it is worth the extra cost on your data.
A practical pilot: 30 days, one workload, real numbers
Do not start with a company-wide migration. Start with one workload you can measure. I have used a version of this approach for AI workflows before, and the full method is in How to Build an AI Workflow for Marketing and Business Productivity.
Week 1: Pick the job and the yardstick.
Choose one well-defined job: meeting summaries, ticket triage, document extraction, first-draft emails, or code review comments. Write down what “good” means before you test anything. Accuracy, format, tone, and time saved are all fair measures. So is “did a human have to redo it?” That last one is the most honest.
Pick two or three candidate models. A sensible starting set in late 2026 is Qwen for general and multilingual work, DeepSeek for reasoning and technical work, and Llama or Mistral where you want a mature ecosystem or European hosting. Add GLM if coding agents are the job. Do not pick on benchmark scores alone. Pick on licence, hosting options, context length, and whether your team can actually run and support it.
Week 2: Run it on your own data, side by side.
Run the open candidates and your current closed model on the same 100 to 200 real examples, with the same prompts and the same evaluation. Blind the review if you can. A human reviewer should not know which model produced which output. Measure cost per successful task, not just cost per call. Include retries, human fixes, and time spent.
Week 3: Test the hard parts.
Security, privacy, and failure modes. What happens with personal data in a prompt? What gets logged, and where? What happens when the model is wrong in a plausible way? Who reviews output before it reaches a customer? Run the same checks you would run for any vendor, because you are now the vendor for this piece of the stack.
Week 4: Decide on routing, not on switching.
Most companies will end up with a mix. Route the well-defined, high-volume work to open models. Keep the frontier for the long, messy, high-stakes work. Revisit the routing every quarter, because the gap is closing fast enough that last quarter’s decision may already be out of date.
A simple routing table to start from:
| Workload | Start with | Keep the frontier for |
|---|---|---|
| Summarising, extraction, classification | Qwen, DeepSeek, Llama | Long documents with subtle reasoning |
| Customer support drafts | Qwen, Llama, Mistral | Complex complaints, regulated advice |
| Coding help and review | Qwen Coder, DeepSeek, GLM | Large refactors, multi-day agent work |
| Private or regulated data | Self-hosted Qwen, Llama, Mistral | Work where you need vendor compliance cover |
| Creative and brand work | Fine-tuned open model | A brand new style or capability, first few months |
This is a starting point, not a ranking. Your data will move rows around. Let it.
The risks, stated plainly
Open models bring real risks, and an honest article should name them before a vendor does.
Security is your job now. The OSSRA 2026 report found an average of 581 vulnerabilities per open-source codebase, and reporting in 2026 noted 175,108 Ollama instances publicly exposed on the internet. An exposed local model server is not a theoretical risk. It is a common misconfiguration. Lock down hosting, authentication, and network access from day one, and treat a model server like any other production service.
Support and operations are your job too, unless you buy them. Open models have no built-in compliance, no account manager, and no service level agreement unless you buy a managed service around them. You inherit responsibility for monitoring, updates, evaluation, and incident response. Budget for that time. The Shopify saving looks different once you count the engineering loop that produced it, and Shopify would be the first to say so.
Benchmarks hide the system. A score reflects the model, the software around it, the context it was given, and how much compute it was allowed to use. Publish one number and you have hidden the other three. The same model can score several points differently depending on who runs the test. That is why the pilot above uses your data, not a leaderboard.
Geopolitics and provenance matter. Four of the five leading open families by adoption in 2026, Qwen, DeepSeek, GLM, and Kimi, come from Chinese labs. That does not make them unusable, and self-hosting changes the data question substantially. It does mean you should know where a model was built, what licence it carries, what data residency your hosting gives you, and whether any restrictions apply to your sector or your government clients. The US government restricted DeepSeek’s hosted app on government devices in 2026 over data concerns. That restriction was about the hosted app, not about self-hosted weights, but it shows how quickly the policy landscape can move. Keep an eye on it, and keep your routing flexible.
The gap is an average, not a guarantee. Open models are close on average, and behind on the hardest work. If your core product depends on that hardest work, the average does not help you. Test the edge cases, not just the easy ones.
What I would do
If I were advising a company starting from scratch today, I would not ask “open or closed?” I would ask “which jobs, routed where, measured how?”
I would start with open models for the bulk of the work, because the cost and control advantages are real and the capability gap for that work is now small. I would keep the frontier for the work that is long, messy, or expensive to get wrong. I would build the evaluation habit early, because the models will keep changing and the only durable advantage is knowing, from your own numbers, what works.
And I would revisit the decision often. A four month gap means the answer you get today has a shelf life. That is not a reason to wait. It is a reason to build a system that can switch, because the next good model may well be open, cheaper, and already downloadable.
That is also, in the end, the same lesson as the small wins I keep coming back to. You do not need a massive rollout to benefit from AI. You need a small, measured win, close to the work, that you can repeat. Open models make those wins cheaper to try, easier to keep, and harder for a vendor to take away. That is worth testing, even if you decide, after the test, to keep paying for the frontier where it earns it.
Frequently Asked Questions
Are open-source AI models really as good as ChatGPT or Claude now?
For most everyday work, close enough that the difference rarely justifies the price gap on its own. For long, complex, agentic work, the closed frontier still leads, by a wider margin. Test on your own workload. Do not decide from a benchmark alone.
Is “open source” the same as “free”?
No. The weights may be free to download, but running them costs compute, engineering time, monitoring, and support. The saving is real for high-volume work, as Shopify’s case shows for serving costs, but total cost includes the work to run the system well. Budget for both.
Can I use open models with sensitive or regulated data?
Often, yes, and sometimes more safely than a hosted API, because you can self-host and keep data inside your own boundary. You still own security, logging, retention, and compliance. The EU AI Act applies to open models too. Get legal and security involved early, and do not assume “open” means “exempt.”
Which open model should I start with?
Start with the workload, not the model. Qwen is a strong all-round starting point, DeepSeek is strong for reasoning and technical work, Llama has the most mature ecosystem, Mistral is strong for European hosting, and GLM is strong for coding agents. Run two or three on your data and let the results decide.
Will open models stay this close, or will closed labs pull away again?
No one knows. The recent trend is convergence, with open capability improving faster than closed capability on Mozilla’s measures. The safe strategy is routing flexibility: build so you can switch models as the field moves, instead of betting the company on one lab’s lead lasting.
Do I need to replace my current AI tools to try this?
No. Start with one workload, run it side by side, and route only what proves itself. Most companies will end up with a mix, and that mix is a strength, not a compromise.
References
Sources checked October 30, 2026. Company cost and revenue figures are reported by the companies or by trade coverage of their claims, and are labelled as such in the text where they appear. Benchmark figures are vendor or independent evaluation results and vary by test setup.
- Mozilla, State of Open Source AI, v1.1, September 15, 2026. Coverage and analysis: https://traictory.com/news/2026-09-18-mozilla-open-weight-gap-4-months and https://www.btin.co.in/chinese-ai-models-cut-frontier-gap-4-months-open-weight-119558
- Innobu, Open-Source AI Models Close the Gap in 2026, October 17, 2026. https://www.innobu.com/en/articles/open-source-ai-models-closing-gap-2026.html
- OmniWire Media, Open-Source AI Models Close on Proprietary Systems in Code Generation Benchmarks, 2026. https://omniwiremedia.com/article/open-source-ai-models-overtake-proprietary-systems-in-code-generation-bench-102
- Unite.ai, Open Models Keep Getting Better. The Economics Are Getting Messier, 2026. https://www.unite.ai/ai-benchmarks-evaluating-enterprise-models/
- Politico Europe, via PPC Land, Ecosia drops Mistral AI for open models, October 6, 2026. https://ppc.land/ecosia-drops-mistral-ai-for-open-models-citing-a-1-year-quality-gap/ and https://madrobot.blog/2026/10/27/ecosia-ai-chat-chinese-models-turn-off/
- PyTorch Foundation, PyTorch Foundation Welcomes Nine New Members, October 20, 2026, including the Shopify continual learning case study. https://pytorch.org/blog/pytorch-foundation-welcomes-nine-new-members-as-momentum-builds-around-open-source-ai-center-of-gravity/
- Open-Source AI News Weekly: Mistral Large 4 Leads, October 2026, including Nous Research funding and Hermes usage as reported. http://newscarpenter.blogspot.com/2026/10/open-source-ai-news-weekly-mistral.html
- Dev.to, What Is Open-Source AI? Why It’s Reshaping the Industry in 2026, 2026, on licensing and adoption. https://dev.to/rashid_1371911653467f5ff2/what-is-open-source-ai-why-its-reshaping-the-industry-in-2026-4okl
