Why token costs don't tell you if enterprise AI works

Enterprise AI is moving through a familiar cycle. Initially, organizations pushed hard, focusing on the largest and most capable models because the technology was exciting and the bill wasn’t there yet. When it arrived, the response was equally predictable: Teams began rationing access, reducing experimentation, and looking for less expensive models.

Ali Ghodsi, co-founder and CEO of Databricks, described the initial phase of this cycle as “tokenmaxxing,” where consuming more tokens is treated as evidence of progress. The most important phase is “valuemaxxing”, which means understanding what that expense produces and directing it towards the work that matters.

This is not simply a new name for cost optimization. Cost optimization asks how an organization can spend less, while valuemaxxing asks how it can get more for what it spends. This distinction will become increasingly important as AI moves from isolated experiments to everyday business operations.

The largest model is not always the right one

Many organizations have started their AI programs using the most powerful model available for almost any task. This approach made sense during the early experimental phase, when teams were trying to understand what the technology could do and model capability was more important than predictability.

As deployments have grown, however, the cost of using a frontier model for every request has become difficult to justify. Some organizations have responded by limiting access or asking teams to do less, but broad restrictions can suppress useful experimentation along with real waste.

A more mature approach is emerging. Teams are starting to pair smaller, specialized models with routine work, reserving more capable models for tasks that require complex reasoning or carry a high cost of failure. Over time, some organizations may also implement patterns within their infrastructure to gain greater control over costs, performance, and sensitive data.

The important question is not whether a large or small model is better. The question is whether the model is appropriate for the job and whether the result justifies the total cost of producing it.

Tokens show activity, not value

Most enterprise AI dashboards are consumption-based. They show how many tokens were used, which models managed them, and how much the organization spent. These numbers are necessary, but they do not reveal whether the work produced a useful result.

A team that consumes double the tokens does not necessarily create double the value. This could be dealing with more difficult work, using an unnecessarily powerful model, or relying on an agent who repeatedly retries failed tasks. A workload with high model costs can still be a great investment if it prevents outages, resolves customer issues, or gives employees hours of valuable time back.

The same problem has existed in observability for years, only with gigabytes instead of tokens. The teams started by gathering everything because maximum visibility seemed to be the safest strategy. As storage and processing bills increased, they began sampling data, reducing retention and eliminating context to control expenses.

In both cases, the response to rising costs has often been to do less. What organizations need instead is the ability to identify which activities are valuable, which are wasteful, and which could become more efficient without sacrificing the bottom line.

The real cost goes beyond inference

A model call rarely represents the complete AI workload. An agent can retrieve documents, query a database, perform telemetry searches, call external services, and generate new applications or infrastructure tasks before producing a response.

Consider an agent investigating why checkout latency increased after a deployment. It could examine logs, compare metrics, inspect traces, retrieve deployment records, and analyze downstream services. If his first theory fails, he can change the time window, search under different service names, and follow several alternative causes.

One human instruction can therefore create hundreds of machine-generated operations. These operations consume compute, storage inputs and outputs, network capacity, and third-party API quotas, none of which can be viewed in your organization’s model spend dashboard.

This means that a seemingly cheap model can support an expensive workflow. A more capable model might cost more per call but reach the correct conclusion in fewer steps, requiring fewer attempts and less support infrastructure. The only way to make this comparison is to look at the complete activity.

Connect activities, costs and results

Valuemaxxing requires organizations to connect three types of information that are often measured separately.

The first is AI activity, including model selection, token consumption, latency, tool calls, retries, and parallel branches. The second is the infrastructure and financial costs created by that activity, including compute, storage, network usage, database load, and external API costs. The third is the result, which should indicate whether the work was completed successfully and whether anyone found the result useful.

Not all results need to be translated into an exact dollar amount. A practical measure may suffice, such as whether an answer was accepted, whether a person had to redo work, whether an incident was resolved more quickly, or whether an automated workflow was completed without intervention.

For example, an incident investigation that generates hundreds of queries can be useful if it helps engineers restore a critical service 30 minutes sooner. A much smaller workflow could still be a waste if it produces a report that no one reads. Token consumption alone cannot distinguish between the two.

Once organizations can connect activity with outcomes, they can make informed decisions about where to use frontier models, where smaller models are sufficient, and where a workflow needs to be redesigned.

Visibility should not require giving up control

Understanding the value of AI requires access to sensitive context. Requests, completions, source code, customer information, and production telemetry can all be part of the same workflow.

Organizations should not be forced to export this information to an external service simply to understand how their AI systems are performing. The insights and production context that make measurement useful could be the same information that security, privacy, and compliance teams can’t allow to leave the organization’s environment.

Business leaders should therefore ask themselves where their AI and observability data is processed, who can access it, how long it is retained, and whether analysis can occur within existing security constraints. Good governance depends on visibility, but that visibility doesn’t have to create a new data control problem.

Keeping analytics close to the underlying data also makes it easier to connect AI activity with the applications, infrastructure and business processes it impacts. This comprehensive view is what allows teams to understand not only how much they spent, but also the return on their spending.

Optimize for value, not volume

Organizations that succeed in the next phase of enterprise AI won’t necessarily be those that use more tokens or spend less. They will be able to see which models and workloads produce meaningful results, then use that knowledge to guide implementation.

This requires moving beyond siled model cost dashboards. Teams must observe the complete path from a user’s request to model calls, tools, infrastructure, and deliverable, while keeping sensitive information under appropriate organizational control.

Tokenmaxxing was a useful step because it encouraged experimentation and revealed what AI could do. Valuemaxxing is the discipline to follow, combining visibility, cost and results so that organizations can continue to experiment without losing control of their expenses or data.

Enterprise AI doesn’t need to shrink simply because the first invoices have arrived. It needs to become more deliberate, with effort appropriate to the difficulty, models appropriate to the tasks, and spending appropriate to the measurable value.