The economics of model routing

GatewayAI engineer, ZürichUpdated 9 November 20266 min read

Frontier list prices fell again this autumn. Anthropic's Claude Opus 5.5 launched in September at $4 per million input tokens and $20 per million output tokens, down from $5 and $25. OpenAI's GPT-6.1 Sol arrived at $2 and $10. Mistral Large 4, released as open weights in October, is priced at $0.70 and $2.10 in preview, with a European deployment Mistral runs itself. The gap between the most and least expensive capable model is still close to tenfold, and that gap is where routing earns its keep.

Where the money goes

Enterprise AI traffic is a long tail of small tasks:

  • Classification, extraction, routing and short rewriting usually make up most requests. A capable lower-priced model does these as well as a frontier model.
  • Drafting and summarisation of medium complexity come next. A mid-priced model is fine; a frontier model is better but rarely worth several times the price.
  • A small share is genuinely hard: multi-step reasoning, long documents, ambiguous judgement. These deserve the best model available.

Frontier models are priced for that last share. Sending everything else to them is where the money leaks.

Three routing rules

  1. By task class. Each agent step declares its class. Classification and extraction default to a lower-priced model, with escalation to a larger one when confidence is low.
  2. By residency. Anything touching personal or regulated data routes to EU-hosted models unless redaction has already run. The rule is enforced in the gateway, not trusted to the developer.
  3. By budget. Teams get monthly caps. When a cap approaches, the router degrades gracefully to cheaper models for non-critical classes, and alerts the owner.

A worked example

This is an illustrative calculation, not a client. Take one million requests a month, each with 2,000 input tokens and 500 output tokens, at the list prices above in US dollars. The routed case moves 60% of requests from Claude Opus 5.5 to Mistral Large 4.

Everything on Opus 5.560% routed to Large 4
Requests per month1,000,0001,000,000
Input cost$8,000$4,040
Output cost$10,000$4,630
Model cost per month$18,000$8,670

That is about 52% less model spend. Whether 60% of your traffic can move without losing quality is an empirical question: run your golden set on both routes before you switch anything. The less visible win is attribution. With every call going through one gateway, each euro maps to a team, a use case and a data class.

Failover is part of the economics

Providers have bad days. A gateway with two or three provider chains retries a failed request on the next provider instead of returning an error to a customer-facing channel, and it records how often that happened.

Routing is not about using cheaper models. It is about spending the expensive ones where they change the outcome.

If you already have model usage spread across teams and cannot say what it costs by use case, that is the first thing a gateway fixes, before a single routing rule is written.

Sources

GatewayEUDACHEnterprise AI
AI engineer, Zürichunus · Applied AI for the European enterprise

Talk to the people who wrote this.