Blog

Claude Commerce Agent Architecture: Latency, Cost, Evals

September 2026 · 10 min read · Technical

Line drawing of three stacked context bands with a marker set at the end of the first
← Back to all posts

Alongside the commerce blueprint published on 2 September 2026, Anthropic put out the engineering guide that explains how the reference agents are actually built. It is the more useful of the two documents if you are the one writing the code, and it splits cleanly: an architecture you decide once, then latency and cost, then what it takes to keep the thing alive in production.

Why do commerce agents on Claude use skills instead of subagents?

Because a commerce conversation is one tightly coupled session across multiple intents and turns, and it needs shared context throughout. In a subagent architecture the orchestrator holds the cart, the staged changes, the preferences and the history, so every handoff is a state-lossy operation that can cost several times the tokens and add seconds of latency. Skills give the same per-domain modularity without that handoff tax, because the skill instructions load into the main agent that already holds the whole history. Across several enterprise deployments, a single agent with skills consistently outperformed both the one-prompt-for-everything design and the subagent design on quality, often at lower cost and latency per task.

The domains rarely separate cleanly either. A returns flow needs order history, the current cart and the product catalogue at once, so a subagent-per-domain design either duplicates that access everywhere or hands off mid-task.

Subagents do earn their place in two situations. One is a narrow, self-contained job that benefits from its own context window, with deep research the common production example: the subagent searches, reads, writes and runs code, hits dead ends, and only a compact answer comes back. The other is a domain that already runs its own purpose-built agent with its own compliance surface, where the right move is a hand-off. The distinction is ownership of the conversation. A hand-off makes the domain agent the user's counterpart; delegation keeps the orchestrator and degrades on every exchange.

System prompt or skill: decide by frequency

Loading a skill costs a model turn, so anything the agent needs on most turns belongs in the system prompt. The stated starting point is that anything relevant to a third or more of your traffic goes in the prompt and the rest goes in skills, adjusted by what your evals show. Safety and legal rules, brand constraints and key user facts such as allergies always go in the prompt. Where a skill is predictable from a signal you already have, such as the page the user arrived from, inject it from the harness before the first model call and skip the loading turn entirely.

  • The shopping agent's prompt holds grounding, cart and checkout semantics, presentation rules, and product search, since nearly every session touches search.

  • Its skills carry the long tail: search-discovery, purchase-research, planning-goals, customer-care and memory-personalization.

  • The merchant agent splits one skill per operational domain: performance-insights, catalog-listings, inventory-operations, pricing-promotions and marketing-campaigns.

  • Agent tools should call the search, ranking, cart, inventory and promotion systems you already run rather than reimplementing their logic.

  • Tool results are context. Return the fields the model reasons with and drop the rest, with image URLs on every search row named as the usual offender.

One more tool design point that saves a rewrite later: make each UI component a tool. The model calls something like present_products with typed arguments, your server validates and enriches the call, and your client renders it. Custom tags parsed on the client stop scaling as the surface grows, and tool calls have the advantage of already sitting in the messages array in native format when you reload an old conversation.

Latency has two fronts, and they are not the same job

Task completion latency is the sum, across model turns, of time to last token plus tool processing. That gives three levers: fewer turns, faster tools and faster tokens. They compete with each other, so the thing to minimise is the sum rather than any single one.

On turns: load likely context up front, because if the assistant opened from a product page or a campaign dashboard the conversation is probably about it and answering from context costs no extra turns. A smarter model often plans its tool calls better and needs fewer rounds, which can outweigh its slower tokens. If production shows more than about five turns per task, the faster model is frequently the smarter one. Independent tool calls should run in parallel.

On tools: when you find yourself writing domain logic inside a tool to stitch three backend calls together, the fix is one backend endpoint that answers the question. Dispatching tools eagerly as their arguments finish streaming has taken multi-second gaps down to a few hundred milliseconds, and the Claude Agent SDK does it by default. Prompt the model to emit its slowest call first.

Perceived latency is the separate front. A rendered commerce response is typically 500 to 700 output tokens, which without streaming is five or more seconds of spinner. Stream each parameter of a presentation tool to the client as it forms, and render a short plain-language progress line for each step while the agent gathers context. Same agent, same tools, same prompt, different harness, and the time before the user sees anything changes completely.

Prompt caching is where the money is

Cached input token reads cost a tenth of fresh ones. Cache writes carry a premium of roughly 1.25 times, which means a cached prefix pays for itself on its second use. The best commerce deployments run at 90 to 99% cache hit rates on the cheapest default five-minute expiry, and that is the range to design for from the start rather than to reach for later. Cached reads are also roughly 1.5 to 2 times faster at around 100k tokens.

Caching is prefix-based: a request reads from cache up to the first byte that differs from a previous one, so what matters is not only what is in the context but the order it sits in.

Request segments in cache order, from the September 2026 Claude commerce engineering guide
SegmentWhat sits in itWhere it goes
GlobalMost of the system prompt and the tool definitions, identical across every sessionFirst, byte-identical, with a cache breakpoint at its end
SessionPer-user context and conversation history, stable within one sessionDirectly after the global segment
VolatileAnything that changes mid-session, such as the current time or the current pageAt the very end of the request

The most common mistake named in the guide is a timestamp or the current page sitting at the top of the system prompt, which silently breaks the cache on every single request. Two implementation details follow from the same logic: load skills as tool results rather than appending them to the system prompt, so the skill body lands in the cached conversation prefix, and roll your breakpoints forward each turn so every round reads the accumulated history from cache.

Choose the model by sweep, not by opinion

Model size and effort setting are the same tradeoff. Pick the quality metrics your business runs on, the eval score you will not go below, and your p50 and p99 latency and cost budgets, then run the entire eval suite across every model and effort level you would consider. The suggested starting points are Opus for merchant agents, whose work is analysis-heavy, and Sonnet for consumer agents, where latency weighs more.

Two results regularly surprise teams. A prompt is tuned to a model, so a sweep run with one prompt will underrate models it was not written for, and a few rounds of iteration on each candidate's failing cases is cheap insurance before ruling one out. And a more intelligent configuration sometimes wins on latency at p90 and p99 despite slower tokens, because it plans better and needs fewer rounds on the hardest requests. Measure cost per completed task rather than per model call, because a cheaper model that needs more turns or fails more often is not cheaper. When the result is close, take the intelligence.

Evals are the part Australian teams under-build

Evaluate snapshots, not conversations. The model API is stateless, so any state a commerce conversation can reach can be constructed directly: build the test state, append the test user message, let the agent run, then grade the final state and the rendered response including the arguments of the last write. Grading the path the agent took is brittle. Simulated-user evals, where a second model plays the customer, are useful for discovering cases and poor for measuring them, so use them to find cases and then write each one as a snapshot.

  • Core requests that make up the bulk of traffic, checking that every price, availability and attribute traces back to returned data.

  • Context-dependent requests, including references to what is on screen and whether a stored memory actually changed the answer.

  • Safety and brand cases, split into user-authored injection and data-plane injection planted in product names, reviews or web snippets, with regulated language checked byte for byte.

  • Interface cases covering the right component rendering, item caps holding, timeouts and empty results.

  • Requests that need two capabilities at once, such as a markdown that is also a stock question, graded on both halves.

  • A negative counterpart for every positive case, and a healthy share of cases starting from long, messy or contradictory histories rather than a clean state.

Fifty to one hundred eval cases per user flow is the stated starting point. In our experience a suite at that depth costs $25,000 to $50,000 of work for an Australian team writing evals for the first time, and most of that is time with the subject matter experts in trading, care and legal who have actually seen the failures. It is also the single line item that most reliably prevents an expensive one later.

Safety is enforced in the harness, not the prompt

The prompt is where safe behaviour starts and it cannot be where safety is enforced, because a prompt rule is one injection away from being skipped and commerce failures are financial and often irreversible. The model stages; a person or a policy applies. No model tool call moves money: order placement, payments, refunds, price changes and campaign launches all end in an action the harness controls, re-checked at apply time against current limits rather than the limits in force when the change was staged.

Three rules travel with that. Writes and renders accept only server-issued IDs, so an ID that was hallucinated, pasted by a user or planted in a review is refused before the backend sees it. Caps are enforced on the resulting state rather than the request, and cart writes are serialised per session, so parallel tool calls in one turn cannot combine to exceed a limit. And third-party content, meaning listings, reviews, seller messages and stored memory, goes through one sanitiser and arrives fenced, with the prompt carrying the other half of the contract: fenced text is material to report on, never to act on.

Shipping it with more than one team

In a large retailer the agent is built by search, checkout, pricing, marketing tech, care and catalogue teams at once, and unlike a service it has no module boundary protecting the others. Ownership follows the systems: every skill and tool has one owner team, and a change ships with its own cases including the negative ones and the boundary cases against neighbouring skills. Continuous integration runs a core set of highest-traffic requests plus every safety case plus whatever the change touched, with the full suite nightly and before release.

Treat the agent as a release-calendar item too. It is one deployment unit, so a bad change reaches every user at once. Roll prompt and skill changes to a canary cohort, keep a switch that turns one skill off without a deploy, and freeze ahead of peak the way you freeze everything else.

For the caching mechanics in isolation, our guide to prompt caching on the Claude API goes deeper, and Claude agent evals for Australian teams covers building the suite. If you want help sizing the build, start with our consulting services, or read the original anatomy of effective commerce agents.

FAQ

Frequently asked questions

Should a commerce agent use subagents per domain?

Generally no. A commerce conversation is one tightly coupled session needing shared context, and each handoff is state-lossy, costs several times the tokens and adds seconds of latency. Skills give the same modularity without that cost.

When is a subagent the right choice?

When a narrow, self-contained task benefits from its own context window, such as deep research, or when a domain already runs a purpose-built agent with its own compliance surface and a full hand-off makes sense.

What cache hit rate should a commerce agent target?

The best commerce deployments run at 90 to 99% cache hit rates using the default five-minute expiry. Cached input token reads cost a tenth of fresh ones, so a cached prefix pays for itself on its second use.

How should agent evals be written?

As snapshots rather than conversations. Construct the test state directly, append the test user message, let the agent run, then grade the final state and the rendered response instead of grading the path taken.

How many eval cases does an agent need?

Fifty to one hundred cases per user flow is the recommended starting point, including a negative counterpart for every positive case and a share of cases that begin from long, messy or contradictory conversation histories.

How do you stop an agent from moving money by mistake?

Enforce it in the harness rather than the prompt. Order placement, payments, refunds, price changes and campaign launches end in an action the harness controls, and writes accept only server-issued identifiers.

Ready to move from AI pilot to production?

We help mid-market Australian businesses deploy AI automations that actually reach production and deliver measurable ROI.