AI Cost Is an Architecture Problem

The unit economics of AI can look almost absurdly good.

If a model call costs very little and saves someone half an hour, there isn’t much of a business case to write.

But I think the economics get more interesting when AI moves from something people choose to use into something systems use automatically.

At that point, consumption becomes an architectural decision.

One user action might trigger one model call.

Or it might trigger an agent that retrieves context, calls a model, uses three tools, calls the model again, retries a failed step, asks another model to evaluate the answer and falls back to a different model if confidence is too low.

The user still did one thing.

The system may have consumed AI many times.

None of those extra calls is necessarily waste.

Retries can improve reliability. Evaluation can improve quality. More context can produce better answers. Agent loops can solve problems a single prompt cannot. Fallbacks can keep a service running.

But they all have economics.

That is why I suspect enterprise AI cost will become as much an architecture question as a procurement one.

The interesting metric isn’t simply cost per token or even cost per model call.

It is cost per useful outcome.

A more expensive model that gets the answer right first time may be cheaper than a cheaper model surrounded by retries. Sending less context may reduce consumption while increasing failure. An evaluation call adds cost but may prevent a much more expensive mistake.

The same architecture also shapes resilience.

Once an AI capability sits inside an important workflow, the organisation depends on more than its own systems. It may depend on a provider’s availability, capacity, rate limits, pricing and service quality.

That means we need to design for what happens when the preferred route isn’t available.

Sometimes that will mean another model. Sometimes a degraded service. Sometimes a human fallback. Sometimes the right answer will simply be that the process can wait.

The important thing is that these are designed choices rather than surprises.

This feels like another reason to think about the organisational harness rather than individual AI applications.

Cost observability, routing, budgets, fallbacks and tracing can become reusable capabilities. Teams should be able to see not just that their AI-enabled service works, but what it consumes to work and how it behaves when a dependency changes.

The model has a price.

The architecture determines how often we pay it, what we get for the money and what happens when we can’t.