AI adoption is on the rise. But is it company performance?

Companies are struggling to turn AI investments into meaningful returns. Usage is growing, but only 13% of organizations report significant transformational value from AI. This gap should push leaders to take a step back and ask themselves whether they are measuring activity or meaningful change.

Under pressure to demonstrate clear AI ROI, organizations often turn to easy-to-report metrics, such as tool usage, active users, or adoption rates. These measures show whether employees interact with the AI. They tell us little about whether the work is improving or whether the investment is generating value.

PwC, for example, found that 64% of workers surveyed have used AI at work in the past 12 months, up ten percentage points year-on-year. At first glance, this is a promising growth sign, but it causes no impact on profits. AI adoption becomes a vanity metric when leaders treat it as proof of transformation. The most effective measure is how AI helps transform insights from previous projects into frameworks, tools and accelerators that improve how teams deliver results.

The hidden problem: AI can accelerate the work that needs to change

Before adding AI, organizations need to understand the purpose of the workflow, identify unnecessary steps, and reconsider how the work should happen. This is important because implementing AI can speed up an inefficient process, moving bottlenecks downstream and creating additional revisions or rework.

That same understanding should guide what organizations measure. Looking at the entire workflow helps teams see how changes in one task affect everything that follows, and whether apparent gains actually reduce effort or simply shift it to someone else.

AI can accelerate individual tasks, such as writing content, summarizing information or analyzing data faster, without improving processes. Faster content generation, for example, can create more work for downstream reviewers and approvers. The bottleneck shifts, even if individual employees report time savings.

This is where productivity metrics can become vanity metrics. Faster accomplishment of a task may seem like a quick win, but it may overlook the effort required to transform the output into a useful result further down the line. Workflow redesign should start with the intended outcome, then determine where the AI adds value and which steps need to change.

Task productivity and workflow productivity are both important. Measuring them together requires teams to look beyond individual time savings to evaluate the quality of the rework and the total human effort involved in achieving the result.

What we learned by becoming our AI customer

At Arionkoder we created Socrates, our internal transformation program, to help us become an AI-native organization. Testing AI as our “case zero” revealed how improvements in the productivity of individual tasks could mask problems across the entire workflow. For example, in our project management workflows, we initially assumed that using AI to generate meeting summaries, action items, and project updates would make those tasks more efficient. However, the increased volume of information still required manual reviews, prioritization, and coordination across teams. This moved the workload downstream, rather than reducing it. We realized that speeding up individual tasks doesn’t necessarily make the overall workflow more efficient. Socrates’ goal remained relevant, but the program needed to help teams move from accelerating isolated tasks to redesigning end-to-end workflows.

This setback became useful when we examined what happened and applied the findings to our next attempt. It reinforced that AI transformation depends on more than just the introduction of the technology. We need to change the way people learn from experiments, make decisions and work together. Socrates needed to help employees connect what they had individually built with how work was done.

The revised approach gives that trial a clear progression. An employee might start using AI to solve a small, real-world problem. The next step would be to see if anyone else could use what they built. From there, the company could look at the broader workflow, identify friction, and connect the work with capabilities developed by other teams. Shared learning among employees helps evaluate whether an improvement remains useful beyond the task it was originally designed for.

This distinction is important. Building something that works for one person is the starting point. To create organizational value you need to understand how others use it and what changes when it becomes part of a larger process.

Our recovery, therefore, began with the expected result. What did the team need to achieve? Which steps contributed to that outcome, and which existed because of an established process that no longer served its purpose? These questions helped shift the focus from adding AI to individual tasks to redesigning the workflow around a clearer goal.

From there, we established baselines and criteria suited to each workflow. We looked at quality, rework and employee engagement, as well as completion times. These measures helped us evaluate improvement throughout the entire process.

At Socrates, we’ve redesigned more than 20 workflows, turning experiments on real business problems into features that others can use. Financial reporting dropped from four hours to 20 minutes, while IT reported 40 percent fewer repetitive tasks and a 60 percent reduction in response times. These results showed operational progress. To understand how these metrics translate into business value you need to evaluate whether effort has decreased in the workflow and how teams have used the gained capacity.

For us, the lesson was to follow improvements all the way through. Saving individual time is important, but it becomes organizational value when it helps the next person do a better job and brings the team closer to the intended outcome.

What should CIOs measure instead?

Tracking improvement for business impact requires more than tracking AI usage or time saved. Leaders must connect adoption with improvements in how work gets done and the results it delivers. Progress should be measured at three distinct levels:

Adoption: Are employees using AI?

Adoption remains useful as a diagnostic measure. Usage, participation in experiments, and employee feedback can reveal where people are building trust and where they need support.

These signals help leaders understand the organization’s readiness, but they do not establish ROI. High usage is a starting point for investigation, not evidence of transformation.
Workflow performance: Is work actually improving?

The success criteria should reflect the intended purpose of the workflow. Depending on the use case, the improvement could mean more precise information, fewer errors, or faster resolution.

Compare performance against a baseline throughout the entire process. Track unnecessary steps eliminated, rework, quality and total employee engagement. Include the people who receive and act on the output generated by the AI. An activity has not necessarily become more efficient if it creates additional revisions or corrections elsewhere.
Business Impact: Is AI-powered workflow improving business outcomes?

Connect workflow improvements to outcomes that leadership already cares about, such as customer loyalty, revenue, quality of service, or cost per problem solved. Producing more content, for example, demonstrates an increase in production. Its business value depends on the content it gets.

Follow the time savings until you use them. Has the additional capacity improved service, supported growth or reduced operating costs? Evaluate value against the total cost of its delivery, including human effort, AI consumption, integration, oversight, training and maintenance.

Together, these measures help leaders track AI adoption to better workflows and business outcomes. If the use of AI increases without reducing human effort, improving quality, or delivering business value, leaders need to reevaluate the approach before scaling it up.

Leaders must redefine AI advancements

Adoption data helps companies see who is using AI and where more support is needed. To understand the value you need to listen to the people doing the work and evaluate whether the gains improve the broader workflow. Leaders must give employees the confidence to point out approaches that add effort without better results, then act on those results. Changing or discontinuing an ineffective approach constitutes progress when learning improves the next decision.

Meaningful transformation manifests itself in eliminating unnecessary work, achieving better results, and gaining greater capacity for work that matters. These are the findings that leaders should use to judge AI progress and guide further investment.