Running an open source model on your own infrastructure gives you control. It also hands you the full weight of keeping that system secure and compliant. For an Australian business operating under the Privacy Act, that responsibility does not pause for weekends, holidays, or busy quarters. The model weights might be free to download, but a secure, audit-ready deployment is a serious ongoing commitment.
This guide walks through what a genuinely secure self-hosted LLM looks like in an Australian context: the controls you need, the regulatory obligations that sit on top, the cost of doing it properly, and the point at which a managed model becomes the lower-risk choice.
What compliance really means for a self-hosted model
Compliance is not a one-time certificate you earn and forget. For a self-hosted model it is a standing set of obligations you have to meet every day the system is online. The moment your model touches customer records, health information, or financial data, it falls inside the same rules that govern the rest of your data estate.
The Privacy Act and the Australian Privacy Principles, which apply the moment you handle personal information
Sector rules such as APRA CPS 234 for regulated financial entities
State-based health records legislation if you process patient data
Your own contractual commitments to clients about where their data lives
Each of these turns a technical project into a governance project. The security work below is the foundation, but the documentation and process around it is what a regulator or an auditor will actually ask to see.
The security baseline you cannot skip
A self-hosted model is a new system holding sensitive data, and it has to be treated with the same rigour as any other. None of the controls below are optional once real customer information is involved.
Authentication on every endpoint, with role-based access control so people only reach what their job requires
Encryption for data at rest and in transit, including prompts, responses, and logs
A patching schedule for the model server, the inference runtime, and every dependency
Network controls that keep the model off the public internet and limit who can call it
Rate limiting and careful input handling to reduce prompt injection and abuse
Identity and access come first
Most breaches start with access that was too broad or never revoked. Tie model access to your existing identity provider, enforce multi-factor authentication for administrators, and review permissions on a fixed cadence rather than when something goes wrong. A model that can read your whole document store is a single compromised account away from a reportable incident.
Meeting your Australian obligations
Security controls protect the system. Compliance is about proving, on demand, that those controls exist and work. For an Australian deployment that means a specific layer of record-keeping on top of the technical baseline.
Audit logging for every sensitive request, retained long enough to satisfy your policy and any regulator
Clear data residency, with the model and its data hosted in an Australian region
Documented controls you can hand to an auditor without a week of scrambling
Retention and deletion rules that match what you have promised customers
An incident response plan that covers the model alongside the rest of your stack
The Office of the Australian Information Commissioner expects organisations to take reasonable steps to protect personal information. With a self-hosted model, you are the one who has to define and evidence what reasonable looks like.
What proper compliance adds to the bill
Here is where honest costing matters. Securing and maintaining a self-hosted model to a compliant standard can add roughly $40,000 a year for an Australian SMB, on top of compute and base staffing. That figure covers security tooling, audit infrastructure, and the engineering time to keep it all current.
Security and monitoring tools, often $10,000 to $20,000 a year
Engineering time for patching, reviews, and audit preparation, easily a third of a senior salary
A staging environment so upgrades are tested before they reach production
Periodic penetration testing and security review, commonly $15,000 or more per engagement
Put against a full self-hosted build that already runs to $80,000 or more a year for a Sydney SMB, compliance is not a rounding error. It is a major line that has to be funded before the system creates any value.
When a managed model like Claude is the safer call
A managed model shifts a large share of this burden to the provider. Infrastructure security, patching, and uptime become someone else's standing job, while you keep responsibility for how you use the tool and what data you send it. For most Australian SMBs in Sydney, Melbourne, or Brisbane without a dedicated security team, that trade is the difference between a project that ships and one that stalls under its own compliance weight.
No model server for your team to secure, patch, or monitor around the clock
Provider-side controls and certifications you can reference in your own documentation
Predictable cost that scales with use rather than a fixed compliance overhead
A faster path to a working, defensible deployment
Self-hosting can be the right answer when volume, data rules, and team capacity all line up. When they do not, a managed Claude build usually reaches the same goal with less risk and a smaller compliance bill. We cost and design both paths honestly for Australian businesses through our Claude consulting services. Book a brainstorm with our Sydney team and we will map the secure option that fits yours.
