You have decided an open-weight model fits a workload and you want it running on Australian infrastructure for data residency. The download is the easy part. This checklist covers what it actually takes to get a stable, compliant deployment in production, in the order you will hit each problem.
Pick your infrastructure
The first choice is where the GPUs live:
Australian cloud regions: AWS Sydney, Azure Australia East, or GCP Sydney keep data onshore with no hardware to buy.
Your own data centre: maximum control and cost discipline at high volume, but you carry all the operational load.
Managed GPU hosts: a middle path, though check where their hardware physically sits before assuming residency.
For most teams, a Sydney or Melbourne cloud region is the sensible starting point. Expect roughly $1,500 to $3,000 a month for a small single-GPU instance and more as you scale up to redundancy.
Stand up the serving stack
A raw model file does not serve traffic. You need an inference layer:
An OpenAI-compatible server such as vLLM, so your existing code talks to the open model with minimal change.
A reverse proxy and authentication so only your applications can reach it, not the open internet.
Autoscaling or a queue if traffic is uneven, to avoid paying for idle GPUs around the clock.
Cover the operational basics
This is where self-hosting earns its reputation for effort. Plan for:
Monitoring of latency, errors, and GPU memory, with alerts that reach a human who can act on them.
A patching schedule for the server, the serving stack, and the model itself.
A rollback plan so a bad update does not take production down at the worst possible moment.
Budget 10 to 20 hours a month of engineering time for this upkeep, every month, on top of the initial two to four week setup. This is the line most business cases forget, and it is the one that usually decides whether self-hosting was worth it.
Prove your compliance
Data residency only counts if you can show it:
Document the region every component runs in, including logs and backups, which often default to an offshore region unless you change them.
Confirm no telemetry phones home to an offshore endpoint.
Keep evidence ready for an audit under the Privacy Act or a client contract.
Before you go live
Run a load test at your expected peak, confirm your cost per request at that volume, and compare it one last time against a managed quote. If the open deployment is not clearly cheaper or clearly required for compliance, a managed model such as Claude may still be the better answer even now, at the end of all that setup work. There is no shame in measuring twice and choosing the simpler option.
The mistakes that cause most incidents
Most self-hosting incidents are not exotic. They cluster around a handful of basics that are easy to skip when you are racing to a demo. The first is logging that quietly defaults to an offshore region. You can run the model itself in Sydney and still break your data-residency story because the logs and backups went somewhere else. Check every component, not just the obvious one.
The second is running without a queue or autoscaling and then meeting a traffic spike. A managed API absorbs a busy morning for you. Your own single GPU does not, and the result is timeouts at exactly the moment the workload matters most. Decide early whether your traffic is steady enough to run flat, or uneven enough to need a buffer, and build for the answer rather than discovering it under load.
The third is having no rehearsed rollback. The model, the serving stack, and the operating system all update, and one of those updates will eventually break production. Teams that practise the rollback recover in minutes. Teams that have only ever rolled forward discover the gap during the outage. None of this is hard, but all of it is work, and pricing that work honestly is what separates a deployment that lasts from one quietly abandoned after a quarter.
The download really is the easy part. Everything in this checklist is ordinary engineering, but together it is a standing commitment of time and attention that a managed model simply does not ask of you. Go in with eyes open, price the upkeep honestly, and a self-hosted deployment can be a sound decision for the right workload. Skip that step and it becomes the project nobody wants to own by the second quarter. If you would rather not carry that, there is no shame in keeping the work on Claude and spending the engineering time on your product instead.
Want a deployment reviewed before it hits production? Book a free brainstorm with us.
