Open the logs of almost any agent in production and count what the model is actually being asked to do. Most of the calls are not writing anything. They are deciding: is this ticket billing or technical, is this lead worth a call, does this transaction look odd. Those are classification problems, and most teams are paying a large language model to solve them one token at a time.
A workshop recap circulating in AI builder communities in September 2026 put rough numbers on the gap. Swapping a routing or scoring step for a small, non-generative classifier cut that step's cost by about five times and its latency by five to seven times, compared with even a budget LLM. Against a full-size model such as Claude, GPT or Gemini, the gap for the same narrow decision is wider again.
How do you reduce AI agent costs without losing accuracy?
Move the routine decisions out of the language model. Put a fast, calibrated classifier in front of Claude to handle routing, scoring and triage, let it return a real probability for each option, and write the thresholds in code. Claude then handles the work that needs language: drafting, reasoning over messy context, and the uncertain cases the classifier flags.
The accuracy point matters as much as the cost point. When you ask a text model to report its own confidence, it produces a number that sounds plausible but is not measured against anything. A dedicated classifier trained on your labelled history outputs a probability you can calibrate and test. That makes the routing logic something you can reason about rather than something you hope works.
The routing logic itself ends up short and readable:
Below the low threshold: the item passes automatically, with no model call.
Between thresholds: it goes to a review queue, or to Claude for a second look with full context.
Above the high threshold: the agent acts on the classifier's answer directly.
Whenever the top two options are close: escalate to a person, because the system is telling you it does not know.
Here is how the trade-off looks for a single decision step, using the relative figures from the workshop recap rather than any vendor price list:
| Approach | Relative cost per decision | Relative latency | Confidence you can trust |
|---|---|---|---|
| Small non-generative classifier | 1x | 1x | Yes, a calibrated probability |
| Budget LLM | About 5x | About 5-7x | Self-reported, not measured |
| Full-size LLM (Claude, GPT, Gemini) | Well above 5x | Well above 5-7x | Self-reported, not measured |
Why so much of an agent loop is classification in disguise
The workshop's estimate was that roughly 90% of the decisions inside a typical agent loop are routing or classification rather than generation. That matches what we see when we audit Claude deployments for Australian clients. An inbound support agent might make eight or nine decisions per ticket before it writes a single sentence: language, product line, urgency, sentiment, whether the customer is a known account, whether a refund is in scope, which queue owns it.
Each of those decisions has a fixed set of answers and years of historical examples sitting in a CRM or helpdesk. That is the textbook case for a classifier. Using Claude for them works, which is why teams start there, but it means paying for a reasoning model to pick from a dropdown.
If you have already worked through the patterns in our piece on token budgeting for Claude agents, this is the next step past matching model tier to task tier: for some tasks the right tier is no generative model at all.
Where Claude Code fits above the classifier
The more interesting part for teams already running Claude Code is what sits above the classifier. Keep the routing thresholds in a small JSON config file rather than buried in a chain of if-statements. Claude Code can then read the decision logs, spot that the review queue has tripled since a product launch, and propose a change to that config as a reviewed pull request.
That split plays to the strengths of each piece. The classifier makes thousands of cheap, fast, repetitive judgement calls. Claude Code does the slower and harder job of noticing when the rules themselves have drifted, explaining why, and drafting the fix. A person still approves the change, and the history of every threshold adjustment lives in version control, which is useful when an auditor or a regulator such as ASIC asks how a customer-facing decision was made.
A config file Claude Code can safely edit
Keep it boring: one file per decision, with the label set, the two thresholds, the escalation rule and a short comment explaining the last change. Give Claude Code permission to edit that directory and nothing else in the decision path. That keeps its changes narrow, reviewable and easy to roll back.
What the saving looks like in dollars
This is an illustrative example, not a client result. Take a Sydney services business whose support agent handles 50,000 tickets a month, with eight classification steps per ticket. If each LLM decision costs about one cent, routing alone runs around $4,000 a month. Moving those steps to a classifier at a fifth of the cost brings that to roughly $800, a saving of about $38,400 a year before you count the faster response times.
The classifier is not free. Expect a one-off build of somewhere around $15,000 to $30,000 for data preparation, training, calibration and monitoring, depending on how clean your historical labels are. Run your own numbers through our ROI calculator and compare them with the full total cost of ownership of an AI agent before committing.
When not to strip out the LLM
This pattern has limits, and it is worth being plain about them. A classifier needs a fixed label set and a few thousand labelled examples. If your categories change every month, or you have no history, start with Claude and log every decision so you build the training set as you go.
Some decisions also depend on reading long, unstructured context: a complaint that mixes three issues, a contract clause, an email thread with a change of tone halfway through. Those belong with Claude. The goal is not to remove the model but to stop spending it on choices a much smaller tool makes better. Our commerce agent architecture breakdown covers how to test those boundaries with evals before you move anything.
A short audit you can run this week
Pull a week of agent traces and tag every model call as either generation or decision. For each decision step, note the number of possible answers, how much labelled history you have, and what a wrong answer costs. Steps with few answers, plenty of history and a low cost of error are your first candidates. Under the Privacy Act, also note which steps touch personal information, since a classifier you host yourself can reduce how much customer data leaves your environment.
If you would like a second pair of eyes on where your agent spend is going, our services team runs this audit as a fixed-scope engagement. Book a time to talk it through.
