When a CIO at a $300 million Australian manufacturer asks whether they should run Claude or stand up their own Llama or Mistral cluster, the question sounds technical. It is not. It is a resource allocation decision that will tie up $640,000 to $980,000 per year in hidden costs if the wrong call is made. This post gives you the honest framework.
The Wrong Question Most Teams Ask
Most evaluations start with 'can we host an LLM ourselves?' The right question is: what problem are we actually solving, and is data residency a genuine legal requirement or an assumption we have not stress-tested? The distinction matters because the answer routes you to entirely different decisions. For most AU mid-market companies in the $100M to $1B revenue range, the data residency requirement is softer than the IT team believes.
When In-House LLM Is the Right Call
There are real scenarios where in-house makes sense. They share a common feature: a specific, written regulatory direction that cannot be addressed by documented data processing agreements.
Your workload involves documents classified at the Australian Government's PROTECTED tier (Defence, ASD, some AUSTRAC or APRA reporting) where cloud data-processing agreements do not satisfy your agency's DTA guidance.
You are a domestic bank or superannuation fund that has received a specific APRA direction requiring that model weights and inference logs not leave Australian soil under any circumstances.
Your legal team has received a formal determination that the data you process qualifies as sensitive information under the Privacy Act 1988, and counsel has ruled that no commercial API meets the threshold, including those with Australian data-centre options.
Those are the real scenarios. If none fit your situation, the case for in-house is almost certainly driven by assumptions rather than legal requirements.
What In-House Actually Costs
Running a modest in-house LLM (say Llama 3 70B or Mistral Large) requires at minimum: two ML engineers at $180,000 to $220,000 each in the Sydney or Melbourne market, a DevOps lead at $160,000, and GPU infrastructure on AWS Sydney at $8,000 to $14,000 per month. Before you build a single internal tool, the annual cost is already in the range of $786,000 to $978,000.
Two ML engineers (salary plus on-costs): $480,000 to $550,000 per year
GPU infrastructure on AWS Sydney (A100 or H100 instances): $96,000 to $168,000 per year
DevOps and MLOps support (lead or contractor): $160,000 to $180,000 per year
Fine-tuning, evaluation cycles, and safety review: $50,000 to $80,000 per year
Total: $786,000 to $978,000 per year before a single internal tool ships to users
Claude API access for a 500-person company with active usage across engineering and operations runs $15,000 to $40,000 per month — roughly $180,000 to $480,000 per year at substantial usage. The cost gap widens as Anthropic releases faster, cheaper model tiers. By the time your in-house team has the cluster running and validated, the hosted option will likely have improved again.
AU Regulatory Context: Data Residency Has Solutions
Most AU companies citing data residency as a reason to go in-house have not checked their legal footing recently. Anthropic processes data through AWS infrastructure with region controls. AWS Sydney (ap-southeast-2) is available today. For Privacy Act compliance, the key test under APP 8 is whether the overseas disclosure is authorised, and in most B2B contexts that test is satisfied by proper data processing agreements with documented safeguards.
APRA's CPG 234 on information security does not prohibit cloud infrastructure. It requires due diligence and a documented risk assessment. AUSTRAC's AML/CTF obligations concern transaction reporting, not model architecture. If your legal team is blocking Claude adoption on data residency grounds, the right move is a scoped privacy impact assessment, not a $900,000 annual infrastructure commitment. AU firms including MinterEllison and Corrs have published APRA and Privacy Act guidance on AI data governance that addresses exactly this question.
Why AU Companies Regret Going In-House
Companies that build an in-house cluster and then pull back twelve to eighteen months later describe the same pattern: the initial build felt like control, and then the model fell behind. Anthropic ships major capability improvements roughly every six to nine months. A Llama 3 cluster that seemed competitive in mid-2024 was two full capability generations behind Claude 3.5 Sonnet by early 2025. The cost of staying current with open-weight models compounds quickly: retraining, re-evaluating, regression testing, updating dependent tooling.
Model capability gaps appear within six to nine months and compound with each major Anthropic release
Fine-tuned models drift over time and require ongoing maintenance; Claude improves without your team's intervention
ML engineering talent is the most expensive and hardest-to-retain technical hire in the Australian market right now
Each new model version requires an internal security review cycle adding four to eight weeks of overhead
In-house models rarely receive the safety and alignment investment that production enterprise systems need
The Decision Framework
Before committing to in-house, put this question to your legal and compliance teams: what is the specific, written regulatory direction, from which regulator, that prevents you from using a hosted model with Australian data-centre options? If the answer is a documented determination rather than a general belief, you may have a real case. In our experience with AU mid-market clients across Sydney, Melbourne, and Brisbane, that document exists in roughly 10 to 15 percent of evaluations. For the rest, Claude is the faster, cheaper, and more capable path.
If your board or risk committee needs an independent read on whether your compliance argument holds, or whether Claude can be deployed within your existing risk framework, book a session with us. We will walk through your specific regulatory posture and use-case fit in 45 minutes.
