Every second AI vendor pitching an Australian business right now says their agents "learn from your team". Almost none of them will tell you where that learning is stored, who approves it, or how to undo it. Warp, the agentic development company, published exactly how its own loop works in late August. The mechanism is unglamorous, checkable, and it gives you a straight set of questions to put to anyone selling you the same idea.
What did Warp actually build on Claude?
Warp runs an AI terminal and development environment used by around 800,000 developers a month, with 56 per cent of the Fortune 500 on the books and more than 400,000 Claude Code sessions running inside it each week. So this is not a lab result. It came out of a problem their own engineers complained about.
Warp's internal code review agent produced unhelpful comments. A first pass that gets 80 per cent of a task right is not 80 per cent useful; it is noisy enough that people stop reading it. The team first tried rewriting the prompt by hand every time they spotted a bad review. That improved output and did not scale. Better context files helped and did not fix it either.
The actual fault was structural. Feedback given to an agent disappears when the session ends. Every run starts from the same blank slate, so the same correction gets made forever.
Their fix has three parts and no magic in any of them:
An inner skill: a plain file holding the domain knowledge and instructions the agent works from. When a pull request opens, the review agent runs off this file.
Human feedback: a person says the output was good, or better, says exactly why it was wrong. Warp captures it where the work already happens, as a comment on the issue or PR.
An outer improver skill: an observer agent that runs on a schedule rather than per task. It pulls the accumulated feedback, compares what the agent suggested against how humans responded, and opens a pull request proposing one small edit to the inner skill file.
A human reviews that pull request, approves it, and merges. The next run of the agent inherits the change. Warp now runs this pattern across its whole open-source repository, with separate spec-writing, review and triage agents each carrying their own loop.
| Part | What it is | Who controls it |
|---|---|---|
| Inner skill | A plain file holding the domain knowledge the agent runs off | Your repository, in version control |
| Human feedback | A comment on the issue or pull request saying why the output was wrong | The reviewer, where the work already happens |
| Outer improver skill | A scheduled observer agent that opens a pull request proposing one small edit | Runs on a schedule rather than per task |
| The merge | A person reviews the proposed edit, approves it and merges | A human holds the merge button |
Is the Claude model itself learning anything?
This is the part that matters for anyone evaluating a vendor claim. Nothing about the underlying Claude model changes. What changes is a text file, tracked in version control, edited by a proposal a person had to approve. Warp founder Zach Lloyd describes the framework as a base domain skill plus an improver skill that refines it, and says the simplicity is the point.
So when an AU vendor tells you their agent gets smarter with use, there is a concrete thing to ask for: show me the file. If they cannot show you an artefact you can read, diff and revert, the improvement is happening somewhere you do not control, and you cannot audit it, roll it back, or take it with you if you change suppliers.
Warp's own guidance draws a hard line between skills and memory. Skills are procedural and stable, changed deliberately. Memory is written automatically by the agent at inference time and never stops moving. Vendors routinely blur those two together and sell the second one using the language of the first.
Five questions for any "agents that learn" pitch
Take these into the next vendor meeting. The answers separate a real loop from a marketing line.
Where is the learned knowledge stored, and can I read it? A file in our repository is a different proposition to a vector store on the vendor's infrastructure.
Who approves a change before it takes effect? If the answer is nobody, the agent can quietly degrade and you will find out from a customer.
How does feedback get captured? If giving feedback is a separate form someone has to remember, the loop starves. Warp's rule is that low friction is what keeps signal flowing.
What happens when the feedback is wrong? Warp's answer is to assume it will be: filter whose input counts, give the agent context to sanity-check against, and keep a person at either the filtering or the final review stage.
How would we roll a bad change back? If the update is a merged pull request, this is a five-minute job. If it is not, ask what the recovery plan actually is.
What does a first agent loop look like for an AU team?
Warp has hundreds of contributors and runs thousands of code reviews. Most Sydney and Melbourne businesses reading this have a team of twelve and one process that everybody complains about. That difference matters less than you would think, because Warp's own finding is that feedback quality beats volume. A small amount of detailed correction from one senior person who knows the domain is worth more than a pile of thumbs-up clicks, because a binary rating never says why.
That puts a first loop within reach of a normal AU services business. Pick one repetitive task where a person already reviews the output: supplier invoice coding, first-pass quote checking, triaging inbound enquiries. Write the domain knowledge into one file. Capture the reviewer's corrections where they already sit. Run the improver weekly, not hourly. Scoping that honestly is a few days of work, small enough to sit inside a A$3,500 engagement rather than the six-figure platform commitments that usually arrive attached to the word "agentic".
One Australian caveat the source does not cover. The moment your feedback corpus contains customer records, complaint text or staff performance commentary, it is personal information and the Privacy Act applies to it the same way it applies to any other system holding that data. Decide where the corpus lives before you start collecting, not after.
What not to conclude from this
Warp's result is real and it is narrower than the headline suggests. Three limits worth holding onto:
This is not set-and-forget. A person approves every change to the skill file. Remove that step and you have removed the only control in the system.
It works best where you can verify the output. Warp's advice is to build the verification harness first, generate a reference set, then let the agent tune against it. Where the domain is not verifiable, restrict feedback to genuine domain experts rather than opening it to everyone.
Improvement is not automatic proof of value. Track the metrics a human already watches, such as time to complete, rework rate and cost, and feed those back in. An agent can get more confident without getting more useful.
The useful takeaway is not that self-improving agents are here. It is that a credible one has a shape you can inspect: a file, a feedback path, a scheduled reviewer, and a human holding the merge button. If a pitch cannot draw you that diagram on a whiteboard, it is not a loop. It is a claim.
If you want a second opinion on an agent proposal already sitting on your desk, or want to scope a first loop on one process, book a time and we will work through it.
