Recent enterprise surveys put data privacy and security as the single biggest blocker to AI adoption, cited by close to 44 per cent of organisations. That figure matches what we hear directly from Australian business owners in Sydney, Melbourne and Brisbane. The models are already good enough for the work. What holds people back is what happens to customer records once they leave the building.
Open-weight models get pitched as the fix. Run the model on your own hardware, the argument goes, and no data ever leaves your network. That is partly true and partly a trap. Self-hosting removes one risk and quietly introduces several others that most small businesses are not staffed to manage.
The barrier is trust, not capability
Ask an owner why their team is not using AI more widely and the answer is rarely that it does not work well enough. It is some version of not knowing where the data ends up. That is a governance question wearing a technology costume, and it explains why open models get so much attention right now. Running a model yourself feels like it removes the question entirely.
To a point, it does. Here is what self-hosting an open model genuinely solves:
Prompts and outputs stay inside your own infrastructure rather than passing through a third party's servers.
No external vendor sees your customer data in transit between your systems and the model.
You control retention and deletion end to end, instead of relying on a vendor's data-handling policy.
What open weights do not solve on their own
Keeping data on premises removes one risk and adds several others. A model running in your own server room is only as private as the access controls, logging and patching wrapped around it. Under the Privacy Act, you are still accountable for how that system handles personal information, whether it runs in a Sydney data centre or on a spare machine under someone's desk.
The parts that still need work, regardless of where the model runs:
Access control, so staff only see what their role actually requires.
Audit logging, so you can show a regulator or a client exactly who asked the model what, and when.
A documented retention policy that meets your obligations under Australian privacy law.
Ongoing patching and monitoring for the underlying infrastructure, which does not stop being your job just because the model is open source.
None of that is free and none of it is fast. Regulated businesses already know this. APRA-supervised financial firms and AUSTRAC-reporting entities carry heavy documentation obligations around model governance and explainability well before AI enters the picture. A small business that bolts an open model onto a spare server without any of that scaffolding is not more private than a well-configured managed service. It is just less visible, and less visible is not the same thing as more secure.
A managed option like Claude, used with a proper data processing agreement and Australian-region data handling, can meet most privacy requirements without a server room at all. For roughly $3,000 to $8,000, a small business can get a governed setup complete with access logging and role-based controls, well under the cost of building and securing self-hosted infrastructure from scratch.
Sort your data before you sort your infrastructure
There is a middle path most Australian businesses miss, because the conversation gets framed as one single choice: send everything to a foreign service, or build your own server room. Neither extreme matches how most businesses actually use their data day to day. The practical move is to sort information by sensitivity first, then match each tier to the cheapest setup that satisfies its own rules.
A simple three-tier split covers most of the small businesses we work with:
General and marketing content: blog drafts, social copy, internal brainstorming. Low sensitivity, so any well-configured managed AI tool is fine.
Business-confidential material: pricing, contracts, internal reporting. This needs a vendor agreement with clear data-use terms and no training on your inputs, but it does not require self-hosting.
Regulated or highly sensitive records: health information, financial account details, anything covered by sector rules layered on top of the Privacy Act. This is the tier where self-hosting, strict access control or keeping a person fully in the loop earns its cost.
Most of what a small business actually asks an AI tool to do sits in the first two tiers. Sorting by sensitivity first stops a business from over-engineering a self-hosted system for low-risk marketing copy, or under-protecting the small slice of work that is genuinely sensitive.
What this looks like for an Australian SMB
Picture a 20-person accounting firm in Melbourne handling routine client correspondence, working papers and internal process documents with AI. Self-hosting an open model to cover that entire workload means specced servers, a part-time systems administrator or an ongoing managed-infrastructure contract, and someone accountable for patching and monitoring. That is an infrastructure build that easily runs to $80,000 to $150,000 in year one, before any of the model tuning work has even started.
The same firm using a governed Claude setup, with the sensitive 10 to 15 per cent of client files handled separately through stricter controls, is typically looking at that $3,000 to $8,000 setup cost plus an ongoing subscription. The privacy outcome for the bulk of the work is comparable. The bill is not.
The honest position is that privacy is a governance problem first and a hosting problem second. Choosing an open model does not discharge your duties under the Privacy Act, it just relocates the work. If data privacy is the reason you have held back on AI, book a session and we will map your obligations against a setup that actually satisfies them.
