Blog

Claude Agent SDK in Production: Lessons From Cresta

November 2026 · 6 min read · Technical

Hand-drawn small terracotta build block beside a much larger testing block filled with check marks
← Back to all posts

Most Claude Agent SDK conversations we have with Australian teams stall at the same point. The demo works, everyone is impressed, and then someone asks what a production build actually looks like. Anthropic has just published a customer story that answers that question more concretely than most, and the parts worth copying are not the parts that make headlines.

Cresta, founded in 2017 and led by CEO Ping Wu, has raised more than $270M and reports more than $100M in annual recurring revenue. Its customers include United Airlines, CVS Health and Marriott. It built Conductor, a natural-language agent builder for customer-experience teams, on the Claude Agent SDK. The Claude blog write-up of the Cresta build is a useful reference architecture for any mid-market team trying to move an agent past the pilot stage.

What did Cresta build on the Claude Agent SDK?

Cresta built Conductor, a tool that takes a customer-experience team from an agent blueprint through implementation, evaluation and optimisation using natural language. The Claude Agent SDK handles context gathering, tool calling and code execution. Conductor layers Cresta's own CX workflows, tools and domain expertise on top. Early deployments reportedly cut initial agent deployment time roughly in half.

The split is the instructive part. Cresta did not rebuild the agent loop. It put its engineering effort into the thing nobody else has, years of knowledge about how contact centres actually run, and left general capability to the SDK. As Xiangru Chen, Cresta's VP of Engineering, put it: "Anthropic gives us a strong, general-purpose foundation for agentic software development."

Three design choices worth copying

Capture expertise as reusable skills

Conductor captures proven patterns, workflows and expert judgement as reusable skill artefacts. Cresta describes this as a knowledge flywheel: each deployment adds patterns the next one can reuse. For an Australian firm, the equivalent is writing down how your best people handle a claim, a complaint or a quote, once, in a form an agent can load, rather than re-prompting it every time.

Decide where rules beat judgement

Conductor helps teams decide where a conversation needs hard business rules and where the model can adapt. That matters most in regulated industries with low risk tolerance. Cresta's guidance is to keep enterprise-critical workflows deterministic. In Australian terms, think of a refund threshold, a hardship conversation in financial services, or anything an ombudsman might later review. Those paths should follow fixed rules, with Claude handling the language around them.

Dogfood before you ship

Cresta used its own tools internally before releasing them. Renjie Li, the engineering lead on Conductor, said: "We used Claude Code internally as a dev tool because it was very effective at the software development aspects of building an agent." Using the thing you build, on your own work, surfaces problems long before a customer does.

The eval habit that answers the model-upgrade question

Every team running an agent in production eventually asks how they know it still works after a model upgrade. Cresta's answer is to rerun a fixed evaluation suite on every new Claude model launch, scored across four axes:

  • Outcome: did the agent produce usable artefacts?

  • Execution path: did it use the right tools and the right context to get there?

  • Quality: how does the output compare with curated reference examples?

  • Resource use: how much time and model cost did the run consume?

The fourth axis is the one most internal teams skip, and it is the one finance will ask about. A new model can be better on quality and still cost more per task, and you want to know that before you switch. Our starter eval suite for Claude agents shows how to set up the first version of this without a dedicated ML team.

Here is how those four axes might translate into checks for a mid-market customer service agent:

Cresta's four eval axes applied to an AU customer service agent, as at November 2026
AxisExample checkWho owns it
OutcomeTicket resolved or correctly escalatedService manager
Execution pathCorrect CRM lookup before replyingEngineering lead
QualityReply scored against gold-standard answersSenior agent panel
Resource useModel cost and time per resolved ticketFinance or ops

Plan for 20% build and 80% hardening

The most useful number in the story is not the revenue figure. It is Cresta's estimate that roughly 20% of the effort goes into the initial build and 80% into testing and optimisation afterwards. That ratio should change how Australian teams budget agent projects.

Most mid-market proposals we review invert it. They allocate most of the budget to building and leave a thin slice for testing, then discover the agent needs months of tuning before staff trust it. As an illustrative example, a $100,000 agent project budgeted on Cresta's ratio would set aside about $20,000 for the first working build and about $80,000 for evaluation, edge-case handling and optimisation. These are illustrative AUD figures, not a quote, but the shape is the point: if your plan has no money left after go-live, it is underfunded.

We cover the build side in more detail in our Claude Agent SDK patterns for Australian production agents, including where the SDK saves time and where you still need your own engineering.

What this means for Australian mid-market teams

You are not Cresta, and you do not need to be. The lessons scale down well. A Sydney insurer, a Melbourne retailer or a Brisbane property manager can apply the same pattern to a single workflow:

  • Put your effort into your own data and expertise, and let the SDK handle the general agent machinery.

  • Mark the steps that must be deterministic before anyone writes a prompt.

  • Write a small evaluation suite on day one and rerun it whenever the model changes.

  • Budget for hardening as the main event, not an afterthought.

If your agent will sit in front of customers, our look at customer service automation beyond chatbots covers the operational changes that tend to matter more than the model.

If you are planning an Agent SDK build and want a second view on scope, evals or budget, our services cover production agent builds, or you can get in touch to talk it through.

FAQ

Frequently asked questions

What is Cresta Conductor?

Conductor is Cresta's natural-language agent builder for customer-experience teams. It is built on the Claude Agent SDK and takes teams from blueprint through implementation, evaluation and optimisation.

What does the Claude Agent SDK handle in a build like this?

In Cresta's build the SDK handles context gathering, tool calling and code execution, while Cresta adds its own CX workflows, tools and domain expertise on top.

How should we test an agent after a Claude model upgrade?

Rerun a fixed evaluation suite on every new model, scoring outcome, execution path, quality against reference examples and resource use, before switching production traffic to it.

How should an Australian team budget an Agent SDK project?

Cresta's experience suggests roughly 20% of effort goes into the first build and 80% into testing and optimisation, so plan most of the budget for hardening after the first working version.

Ready to move from AI pilot to production?

We help mid-market Australian businesses deploy AI automations that actually reach production and deliver measurable ROI.