A finance team member opens the latest AI invoice and sees the same thing as last month: another, even bigger overage. Perhaps that is to be expected given that the company is going all-in on using AI tools. Now, where is that AI cost-benefit analysis document?
In their rush to embrace AI, companies encouraged employees to adopt online AI tools, such as ChatGPT, Gemini, and Claude. The technology quickly proved valuable for many tasks, especially routine and repetitive work. Some managers eagerly eyed this cost savings opportunity by replacing people with technology. Then reality set in. AI mistakes (“hallucinations”), frustrated customers, and loss of institutional knowledge proved costly. Consequently, some companies backtracked and replaced technology with people. The AI value just wasn’t there.
The cost of using online AI tools has come down dramatically in the past year, yet companies are seeing their monthly AI bills increasing. With increased AI adoption, you expect the total costs to go up. Nevertheless, there are numerous examples of companies being shocked by outrageous bills.
The unit of cost of these tools, the humble token, became a productivity metric. The more AI you used, the more tokens you consumed and, thus, the more productive you must be. So went this flawed reasoning. What was missing was the value. If the AI was being used to get more work accomplished, or achieve better quality outcomes, then it might be justified. But as a productivity measure, it was a disaster. The loophole, of course, is that employees quickly figured out that sending every task to the AI would increase their perceived productivity. There’s a name for it: tokenmaxxing. Unbridled spending yielding unclear value. For example, if you have a person who delivers packages, would you ever reward the driver that spends the most money on gasoline?
With the tokenmaxxing hysteria now in the past, companies are wrestling with AI being a new major cost item on their financial statements. But there are ways to bring down the cost, the most obvious is by matching an expense to its value. The problem is that employees are running every task, big or small, through the most capable AI model available without stopping to ask whether the task needs that much horsepower. The interface looks the same whether you’re asking a chatbot to summarize a two-page memo or to reason through a hard technical problem. The bill does not look the same. Now multiply that by a workforce of a few hundred employees. Add AI agents that can burn through hundreds of thousands of tokens on a single task. It’s time to right-size a company’s token usage by matching the task to the tool.
Frontier models, the newest and most capable systems, earn their premium price when the work is genuinely hard: complex research, technical reasoning, and work that requires synthesizing large amounts of information. It’s also a small slice of what most employees do with AI on any given day.
A lot of everyday workplace AI use is closer to administrative than intellectual. Summarizing a document. Pulling a name out of an old email. Reformatting notes. None of that requires a model built to compete at the edge of technology. It requires a tool that’s fast and cheap. You don’t hire a surgeon to put on a Band-Aid, but plenty of companies are doing exactly that with AI, and paying surgeon rates for it.
Set the cost question aside for a second. Every document, spreadsheet, or transcript you feed into a cloud-based model leaves your machine and lands on someone else’s servers. For routine internal tasks, that can create a privacy and governance question the company didn’t need to create in the first place. Contracts, client notes, internal strategy documents: all sent to a system you don’t control to do work your own laptop may have been able to handle locally.
Modern laptops have enough computing power to handle routine AI workloads locally. For the right tasks, companies can use hardware they already own instead of paying to send every query to a remote data centre. Start by looking at workloads, not licenses: Which tasks require sophisticated reasoning? Which are simple retrieval or summarization? And which involve data that shouldn’t leave the organization unnecessarily?
Before renewing a company-wide AI license, ask a blunt question: how many employees actually need frontier-level reasoning? For most organizations, only a fraction of employees regularly need the expensive tool.
The cost fix isn’t complicated. Save the frontier models for the problems that genuinely require them. Push the summarizing, sorting, and searching that makes up most of the daily grind onto tools that can run locally and without shipping data anywhere it doesn’t need to go. Do that, and the bill finally matches the value produced.
Jonathan Schaeffer is an AI pioneer, computer scientist, and entrepreneur. He is Distinguished University Professor Emeritus of Computing Science at the University of Alberta and one of the four co-founders of the Alberta Machine Intelligence Institute (Amii). He is the founder of Synsira Software Solutions and creator of Kind, a desktop application that provides users with private, accurate, and safe AI experience. His current work focuses on practical, responsible uses of AI and the long-term implications of how intelligent systems are built and deployed.
Leave a Reply