Skip to main content

Do we have an AI token cost solution in sight?

Updated 2 min readMarkdown
Do we have an AI token cost solution in sight?

In 2026, AI token cost has become a growing concern in boardrooms across the world.

As a solution, one approach that is getting more attention is model routing - sending each AI task to the model that makes the most sense for that particular task.

Why use an expensive frontier model for a routine task when a much cheaper model can do the job?

Routine, lower-complexity tasks can potentially be routed to open-weight models including models originating from China where pricing can be 5x–25x lower than comparable frontier models.

There is another interesting advantage.

Organizations can run these Chinese open-weight models on their own infrastructure or access them through routing platforms such as OpenRouter. This can potentially give organizations more control over data residency, compliance and governance than directly sending sensitive workloads to an externally hosted Chinese model - which has been a major concern for organizations worldwide.

And the change in adoption numbers is quite remarkable.

Recently released data from OpenRouter indicates that Chinese-origin open-weight models accounted for approximately 46% of routed token volume in May 2026, compared with approximately 34% for US-origin models.

Compare that with October 2025 Chinese-origin at ~28% vs. US-origin at ~62%

That's a significant shift in just a few months.

But there are some important caveats

  • OpenRouter is not the whole market. It is a proxy for model-routing activity, with a developer heavy and API-centric user base.

  • More token volume doesn't necessarily mean more valuable workloads. Chinese-origin models could simply be handling a larger share of routine or lower complexity tasks.

Still, the direction is interesting.

As Chinese models continue to improve, the competition may not remain only about cost and volume. It could increasingly become about capability as well.

Hopefully, over the coming quarters, the boardroom's concern about AI token cost will shift to something else and that would be a good problem to have.

About the author

  • Engineering Leadership executive with a proven history of managing global distributed teams. Working on driving the next-generation Cloud-Native architecture and Agentic AI development across SaaS platforms, while focusing on continuous delivery, product quality, and strategic talent development.
RRahul R1mo ago

Value vs. Volume: If 46% of token volume goes to lower-cost models, what actual percentage of organizational spend does that represent, and are we measuring ROI on token counts or business outcomes?

Those are valid questions and answer to those can be more informative. OpenRouter who publised the trend didn't really provide the detailed level information.

The cost per successful task is the part that matters. A cheap model that needs two retries can get expensive fast.

Routing helps, but a lot of the waste I see is upstream of the model choice. Huge system prompts, full chat history sent every turn, no caching. Trimming that is boring but often cuts more than switching models. Has anyone here seen routing hold up once you count the extra eval work for each model?

Rishita Sharma Those are best practices. Some of that can be automated and some are guidelines for developers to be followed as part of their day to day work. Sorry, cannot get the eval question.

The routing argument lands, but the comment about measuring cost-per-successful-outcome instead of cost-per-million-tokens is the part I would build on. A cheaper model that fails and triggers a retry or an escalation is not actually cheaper, so routing needs a success signal in the loop, not just a price table. When you route to the cheapest capable model, what defines 'capable' at decision time, a prior eval score or a live confidence check?

@kartik-nvj Thats a valid point. Successful-outcome is something which would build the credibility and not just the cost. Havings said that there's a merit in routing the low complexity tasks to the cheaper models if one is able to get the confidence in prior evaluation. And capability has to be understood through both prior eval score and refinement of model deployed to live.

Model routing is probably more useful when treated as a workload optimization problem rather than simply a way to pick the cheapest model. A 10x reduction in token price can disappear quickly if the cheaper model needs more retries, longer prompts, or additional validation to reach the same task-level outcome.

In production AI work at IT Path Solutions, I’d measure routing decisions against cost per successful outcome, latency, failure rate, and escalation frequency not just cost per million tokens. That also makes the open-weight vs. frontier-model comparison more meaningful: the right question is which model delivers the required quality and reliability at the lowest total cost for that workload.

The interesting next step is dynamic routing based on observed task performance, where the router learns which model is actually economical for each class of work rather than relying only on predefined model tiers.

Absolutely! Token cost is only one part of the equation. Engineering teams certainly need to observe other engineering parameters while integrating the open-weight models.