The boom in free artificial intelligence chatbots has given users a taste of powerful technology at no cost, but the companies behind those systems are spending hundreds of billions of dollars to build and run them. Microsoft, Google and Anthropic are among the firms investing heavily in large language models, the technology that powers assistants such as ChatGPT, Claude and Gemini. Offering basic access for free is a deliberate strategy to encourage adoption, while paid tiers promise extra capabilities for coding, billing and other tasks.
Yet for businesses trying to build their own AI products, or integrate AI agents into their operations, the economics have become awkward. Setting a price for these services is surprisingly difficult because the costs involved are hard to predict. Simon Gooch, an executive at identity management firm Saviynt, says committing customers to a fixed cost model for a year or more “doesn’t make any sense” when the underlying economics are still evolving.
The uncertainty stems from how large language models work. When a user submits a prompt, it is broken into tokens, small mathematical units the model processes to generate a response. Those responses are converted back into text or computer code. But the process is not deterministic: slight wording changes can produce different outputs, the same prompt may yield different results, and different models handle tasks differently. With agentic AI, where multiple agents collaborate to make decisions, token consumption and unpredictability grow further.
Recent trends have made matters more confusing. Goldman Sachs analysis shows that while the price of individual tokens has fallen sharply, overall consumption has exploded as companies and consumers rely more heavily on AI. The bank projects token usage will increase twenty-four-fold between 2026 and 2030, reaching 120 quadrillion tokens each month as agent-based systems become common. Many organisations have only a vague idea of how many tokens they are using until the bill arrives. Reports suggest Microsoft has curbed its engineers’ use of some third-party coding tools, and Uber exhausted a year’s worth of AI coding tokens in just a few months earlier this year.
Will Venters, an associate professor at the London School of Economics, says firms are often caught off guard in internal experiments as staff consume tokens without clear cost visibility. “People are finding it really hard to manage that cost,” he says, noting that because outputs are non-deterministic, so is the value. Businesses are trying workarounds. Oliver King-Smith, founder of engineering software firm smartR AI, says smaller companies can “fly under the radar” using flat-fee personal accounts that larger vendors likely dislike. But he warns this cannot last, as the big platforms are losing money on those accounts and will eventually clamp down once shareholders demand profits.
Companies are also being urged to choose models more carefully and write clearer prompts. Rob Steele, CFO of UK accounting software firm iplicit, argues users should specify what they want just as they would give detailed shopping instructions. The unpredictability multiplies when AI is embedded in a product rolled out to thousands of users. Venters points out that AI costs can balloon when extra tokens are needed for testing, security or guardrails, and adding more AI agents is as easy as clicking a button, unlike hiring human staff.
Despite the volatility, Venters cautions that higher token use can bring more value, comparing it to a tool that becomes more useful with more use. The difficulty for companies is passing those costs to their own customers. Bill Peterson, senior director of product marketing at Sumo Logic, says “nobody’s really figured it out.” His firm is previewing agentic AI security services and debating whether to raise prices broadly, charge by results, or offer bundles of incidents. Any choice could be upended if large language model providers change pricing again. “Customers don’t like that. That’s not how anybody builds a budget,” he says.