The AI bill is becoming a management discipline

The Linux Foundation’s new Tokenomics initiative confirms what fast-scaling AI companies are already discovering: model usage is no longer an experimental expense. But counting tokens is only the beginning.

For most startups, the first AI bill feels almost charming.

The product is still an MVP. There are a few hundred users, a handful of prompts, and enough startup credits to make frontier-model inference look nearly free. An engineer can solve an extraction problem, a routing problem, and a classification problem without designing a schema, training a model, or building a data pipeline. The model absorbs the ambiguity, the feature ships, and the company moves on.

Then the product works.

One user action becomes five model calls. Prompts acquire retrieved documents, conversation histories, validation passes, and retry loops. Agents continue consuming tokens after the user has stopped typing. What looked like a software subscription begins behaving like a variable input cost.

In June 2026, the Linux Foundation announced its intention to launch the Tokenomics Foundation, working alongside the FinOps Foundation to develop open standards, benchmarks, and practices for measuring the economics of AI infrastructure. Its initial supporters include major AI buyers and suppliers such as Google Cloud, Microsoft, IBM, JPMorganChase, Salesforce, SAP, and ServiceNow.

This is less a new technology announcement than an admission that AI has acquired a new organizational problem.

The cloud bill, reinvented

The evolution resembles cloud computing.

Cloud began as a faster way for developers to provision infrastructure. It subsequently created a variable and often poorly understood expense that crossed product, engineering, and finance. FinOps emerged to make that consumption visible, attributable, and governable.

AI is now moving through the same cycle, except the economic unit is stranger.

A cloud bill generally maps onto recognizable resources: machines, storage, bandwidth, and database capacity. A token bill may represent a customer question, a document copied into context, hidden reasoning, an agent retry, a cached prompt, a generated report, or a failed attempt to produce valid JSON.

The FinOps Foundation says managing token cost and consumption in SaaS-model AI is now the top challenge identified in its practitioner survey, citing fragmented purchasing, opaque billing, inconsistent allocation, and pricing that varies across models and use cases.

Meanwhile, the Linux Foundation’s announcement cites a forecast that global monthly token usage could increase 24-fold between 2026 and 2030, reaching 120 quadrillion tokens. Whether that precise forecast proves correct matters less than the mechanism behind it: agentic applications consume sequences of model calls rather than isolated chat responses.

A recent account of enterprise AI cost management described firms such as Priceline, Qualcomm, and Bristol-Myers Squibb using spending caps, dashboards, showback, and chargeback to manage AI expenditure. The same report noted an estimate that an agentic task can consume roughly 50 times the computing resources of a conventional chatbot interaction.

The AI bill is therefore not simply growing because models are expensive. It is growing because software is being redesigned to consume intelligence continuously.

The invoice hides the computation

The monthly invoice may say:

LLM usage: $84,217

But “LLM usage” is a billing category, not a category of work.

These computations do not have the same technical requirements or economic value.

A frontier model may be indispensable for a complex synthesis task and wildly excessive for converting a date into ISO-8601. Yet they can appear beside one another on the same provider invoice, charged in the same unit.

This is the limitation of token-level FinOps: it can tell an organization where the money went, but not necessarily what computation it purchased.

That distinction matters for executives because there are two very different ways to reduce an AI bill.

The first is to make the existing architecture cheaper.

The second is to change the architecture.

The prevailing cost-management stack

Today, a technically capable company can assemble a credible LLM cost-management system from existing tools.

Langfuse can capture hierarchical traces across model calls, tools, retrieval, and custom code, then associate them with token usage, latency, cost, users, and sessions. It also provides evaluations, datasets, and prompt management.

LiteLLM can provide a unified gateway across more than 100 model APIs, with spend tracking, budgets, routing, load balancing, and caching.

Portkey offers similar gateway functions, including model and provider fallbacks, semantic caching, conditional routing, budget limits, and guardrails.

Together with manual engineering, this stack can address much of the obvious waste:

Research supports this approach. RouteLLM reports cost reductions above 85% on MT-Bench, 45% on MMLU, and 35% on GSM8K while retaining approximately 95% of GPT-4 performance in those benchmark settings. FrugalGPT reports that model cascades can match the strongest individual model in its experiments with cost reductions of up to 98%. These are benchmark results rather than guarantees for production workloads, but they demonstrate that request difficulty is heterogeneous and that not every query needs the strongest available model.

For many companies, this is sufficient.

A team whose main problem is an expensive default model, duplicated context, missing rate limits, or obvious repeated queries does not need program synthesis. It needs observability, budgets, caching, and competent routing.

The missing middle

The unresolved problem begins after a dashboard identifies an expensive workflow.

Suppose an organization discovers that it spent $40,000 last month on a prompt that reads support conversations, extracts an account identifier, classifies intent, checks an SLA rule, normalizes dates, and emits a JSON record.

The existing stack can show that the workflow is expensive. It can cache exact repeats or route the whole prompt to a smaller model.

But the prompt is not really one task. It is a compact description of a backend system that was never built:

Some stages require language understanding. Others are lookup, arithmetic, formatting, or business rules.

Plotting workflows by volume and ambiguity makes the priority obvious — the high-volume, low-ambiguity corner is where architectural work pays off first:

The conventional response is to assign engineers to reverse-engineer the prompt, define the contract, build the pipeline, construct an evaluation set, add confidence thresholds, shadow-test the replacement, and maintain it as the workflow changes.

That work can take weeks or months.

The biggest competitor to a product such as Seldon is therefore not one particular startup. It is a good AI and data engineering team using Langfuse, LiteLLM or Portkey, a cache, and some disciplined manual work.

The question is whether that work can itself be automated.

Beyond routing

Seldon began with the language of model routing because the gateway is a useful insertion point. It provides access to production traffic, a place to observe traces, and a mechanism for shadow deployment and fallback.

But routing is not the deeper product.

A conventional router asks:

Seldon’s longer-term question is:

In mature systems, the frontier model moves from the hot path — where every request pays for it — to the exception path:

The intended loop is:

This is closer to automated backend productionization than token arbitrage.

It borrows from program synthesis: input-output examples provide a partial behavioral specification, while the search space consists of typed components such as parsers, classifiers, extractors, retrievers, normalizers, and validators. The generated program does not need to replace the entire workflow. It only needs to handle a measurable, high-confidence slice economically, while uncertain cases fall back to the original model.

Academic work offers evidence for pieces of this thesis, but not yet the complete system. “Compiled AI” explores generating executable artifacts during a compilation phase so that subsequent executions no longer require model calls. Other recent work describes extracting deterministic, lower-cost workflows from agent traces rather than merely making each inference cheaper. These directions support the concept, but they do not prove that arbitrary enterprise prompt traffic can be safely converted into production software.

That proof must come from deployments.

What Seldon must prove

The distinction sounds compelling in a diagram. It is much harder in production.

To offer a genuine advantage over the existing stack, Seldon must demonstrate that it can complete the entire loop, not merely produce another audit:

1. Discover a meaningful recurring workflow automatically.
2. Infer the correct contract rather than blindly imitating noisy LLM outputs.
3. Decompose the workflow into appropriate ETL, ML, NLP, and reasoning components.
4. Generate an implementation that engineers can inspect and own.
5. Evaluate it using task-specific metrics.
6. Know which cases to refuse.
7. Deploy gradually with fallback and rollback.
8. Maintain the replacement as schemas and distributions evolve.

Until that happens repeatedly, Langfuse plus a gateway, caching, and manual engineering remains the safer and more complete solution.

This is not a weakness in the thesis so much as a precise statement of the opportunity.

The executive playbook

Executives and founders scaling AI products should not begin with the question, “How do we reduce tokens?”

They should begin with four others.

1. What business outcome does each workflow produce?

Move beyond cost per token or cost per call. Measure cost per resolved support case, processed invoice, completed research report, or qualified sales opportunity.

2. Which workflows concentrate the spend?

Most organizations will find that a limited number of workflow clusters account for a large portion of inference cost. Those clusters deserve architectural review.

3. What work is actually inside each call?

Decompose prompts into reasoning, extraction, classification, retrieval, transformation, validation, and formatting. Do not assume the entire request belongs on one model merely because it was originally written as one prompt.

4. What is the lowest-cost implementation that clears the quality bar?

Test alternatives in increasing order of flexibility. The pragmatic architecture is tiered — cheap, deterministic tiers handle the bulk of traffic at the base, and the frontier model is the narrow tip reserved for genuine ambiguity:

Or, more operationally, climb only as far as the contract requires:

The objective is not to push every workflow downward. It is to find the lowest layer that can satisfy the business contract safely.

Tokens are the start of the accounting

The Tokenomics Foundation is an important signal because industries create standards when improvisation becomes expensive.

It should give companies a better vocabulary for measuring AI consumption across providers, models, and deployment options. But the most consequential question sits one layer above the token:

The cloud era taught companies to attribute infrastructure spend to services, teams, and customers. The AI era will require them to attribute inference to workflows, outcomes, and architectural choices.

Many successful workflows follow the same maturation arc — from exploration served by the frontier model to production served by deterministic, compiled code:

Some of those workflows will remain model-native. Others will mature into ordinary software.

The next generation of AI cost management will not merely find a cheaper token. It will identify when tokens are no longer the right implementation.

That is the transition Seldon is being built to automate.