Gartner: AI Inference Costs Expected to Increase 5x by 2028

The Inference Paradox: AI Costs Expected to Explode by 2028

Imagine you are driving a car that becomes more fuel-efficient with every kilometer you drive. Sounds great, right? But at the same time, the price of fuel skyrockets. This is somewhat what is happening with artificial intelligence, according to Gartner. They predict that AI inference costs per agent workflow will increase more than five times by 2028. That's right, five times!

The issue is that while AI models are becoming cheaper and more efficient, the complexity of workflows is increasing. This means that even with cheaper tokens, the total cost of AI is rising. This is what Gartner refers to as the "Inference Paradox." We improve the economy per unit, but the overall cost of AI continues to rise without a clear proportional return.

The Product Leaders' Dilemma

Product leaders are at a crossroads. They cannot simply rely on token efficiency to justify AI costs. Each new generation of AI requires more tokens, and often, more expensive tokens. There is no universal economic model that solves this. To remain competitive, companies will have to develop and maintain complex multimodel ecosystems.

Gartner highlights three fundamental trends driving this token economy. First, the costs of foundational models are improving rapidly. Second, AI efficiency is enabling the use of more powerful and expensive models. And third, more sophisticated workflows use many more tokens than simple chatbot interactions, which elevates inference costs.

The Race Against the Cost Curve

In practice, this means that innovation is outpacing the cost curve. Tokens are becoming more efficient, but not as quickly as AI capabilities and the associated costs. It's as if we are running on a treadmill that accelerates faster than we can keep up with.

The paradox becomes evident when we compare a simple chatbot with an AI agent. While the chatbot merely reads and responds to queries, an AI agent needs to reason, negotiate, and constantly question. All of this increases inference costs. In comparison, a reasoning agent model can increase the provider's inference costs by at least five times, and this number only grows with task complexity.

To ensure a return on investment in advanced AI, such as reasoning agents, it is necessary to achieve exponentially greater returns compared to basic models. This or drastically optimize classification, routing, and orchestration of inference to calibrate complex tasks against more economical intelligence. Both outcomes are possible, but require significant effort in complex workflows.

Challenges and Paradoxes of AI

What is clear is that AI is not a linear path of progress. Costs are not just a matter of token efficiency, but of how we integrate and manage these technologies in complex systems. Gartner issues a warning: relying on generic autonomous intelligence can result in uncontrolled costs, much higher than those of optimized product ecosystems.

Gartner's forecast is a reminder that as technology advances, challenges also multiply. The future of AI is promising, but not without its own paradoxes and dilemmas. And that is exactly where the beauty and complexity of this technological revolution lie.