By Saul Bowden
What does an AI token represent?
There are a few different definitions for an AI token:
- A unit of language – AI breaks texts into tokens so it can mathematically predict what word is most likely to come next.
- A unit of context – Represents every input into the model: your prompt, instructions, documents, and chat history. This consumes input tokens; the model’s answer consumes output tokens.
- A unit of compute – Every token has to be processed through a model. More tokens usually mean more GPU work, more memory usage, higher latency, and higher costs.
- A unit of billing – Most AI providers charge by tokens; this is a sum of the various costs it’s taken to generate your output, and whatever margin the provider has built in.
For our purposes, we want to focus on number 4. We are talking about a token serving as the smallest practical billing and processing unit for AI output.
This means that you’re paying a combination of AI infrastructure costs, model licensing fees for AI wrappers, overhead, and, of course, margin. The problem here is that it makes it very difficult for users to understand exactly how far their monthly token allowance will actually take them.
The nature of tokens also makes this more difficult. A token is the unit a model uses to process an input and output, and in commercial terms, the unit used to meter compute consumption. The compute cost behind that token can vary dramatically depending on the model, the size of the context window, the complexity of the task, whether reasoning is involved, and how many times an agentic workflow calls the model.
This opacity benefits AI companies that use allowances, such as Copilot. GitHub’s current docs say paid Copilot plans include monthly GitHub AI Credits and that users can pay extra once those credits are exhausted. GitHub also says that additional usage is billed at per-token rates, with 1 AI credit = $0.01 USD, but users can’t reliably predict how much usage their specific task will require in advance.
An AI company does not necessarily need to raise the subscription price to change the economics for users. It can change the meter: the included models, the multiplier, the credit system, the token rates, the context rules, or how heavy usage is routed. The result is the same: heavier token and compute usage gets passed through more directly, while users still have limited visibility into how much a specific task will actually consume.
How Can Companies Limit Token Burn?
This represents a significant risk for AI companies. They need tokens to run their operations, but the underlying infrastructure can change its pricing in a wide variety of ways that could completely disrupt their underlying margin, pricing, reliability, and business model.
This doesn’t mean that companies are at the mercy of LLM providers. There’s a variety of levers that can be pulled to limit exposure. A classic example is Lovable, which utilizes a Dynamic Model Orchestration mix of models for different tasks. Simpler models handle user context, while more advanced models do the heavy lifting in order to generate outputs. This is complicated to do, but it certainly limits costs.
From an infrastructure perspective, there are other things you can do as well. Rather than relying entirely on generic AI SaaS tools or hosted frontier-model APIs, companies can build smaller, task-specific models or systems. This gives you the ability to select specific providers, for example, Neural Edge, who can offer the exact infrastructure specifications you need for your use case.
This gives you far finer control over your exact AI infrastructure. Companies like Strates Infrastructure Consortium Inc. can work with you to determine the exact solution needed for your specific use case. Ultimately, this approach provides more clarity over long-term costs and helps to limit the number of risks associated with AI. You know where your data is going, you get more visibility and control over your infrastructure costs, and exactly what specs you need to run it.