Accounting firms handle exactly the kind of data clients expect to stay private: financials, tax file numbers, payroll. So when partners hear that open-source AI can run entirely in-house, it sounds like the obvious safe choice. The reality for most Australian firms is more practical, and usually cheaper than the in-house instinct suggests. Here is a grounded guide for partners weighing it up.
What the work actually needs
Most AI use in an accounting or bookkeeping practice falls into a few buckets:
Drafting client emails, engagement letters, and plain-English explanations of tax positions.
Summarising statements, reconciling notes, and first-pass categorisation of transactions.
Answering staff questions about ATO guidance and internal procedure.
None of these require a frontier-scale model running on your own hardware. They require accuracy, a sensible tone, and care with confidential data. That is a process question as much as a technology one.
The open-source temptation
Running an open model in-house is appealing for a clear reason: client data never leaves the firm. That genuinely matters for confidentiality and for some professional obligations. But the costs are real:
A production server runs roughly $1,500 to $5,000 a month for a small or mid practice.
Someone has to patch, monitor, and update it, which is 10 to 20 hours a month most firms cannot spare from billable work.
A misconfigured server is its own data risk, arguably larger than a reputable managed provider with a security team behind it.
For a 30-partner firm with IT staff, it can work. For a five-person suburban practice, it rarely does, and the partners end up maintaining infrastructure instead of serving clients.
The managed alternative
A managed model such as Claude, with the right data terms, gives most firms what they need without the server:
No-training data terms mean client information is not used to improve the model.
Australian data residency options keep records onshore.
Safety tuning lowers the odds of a confidently wrong tax answer reaching a client.
A typical small firm spends $200 to $1,000 a month here, with no infrastructure to manage and partner time freed for the work clients actually pay for.
A sensible policy either way
Whichever path you choose, put guardrails in writing:
Never paste a client's full identifying details into any tool without checking your data terms first.
Keep a human review step on anything that goes to a client or the ATO.
Record which tool is approved for which task, so staff are not improvising with whatever is open in a browser tab.
Most Australian accounting firms get the best result from a managed model plus a clear usage policy, and revisit open source only if volume or a specific obligation demands it. The policy matters more than the model: a careful firm on a managed tool beats a careless one on a private server every time.
What a sensible first year looks like
For most Australian practices, the path that works is unglamorous and cheap. Start with a managed model and a one-page usage policy. Pick two or three jobs that eat the most admin time, such as drafting client emails or summarising statements, and run those through the tool with a human checking the output. Measure the hours saved over a quarter rather than guessing at them.
Only once you have real usage data does the open-source question become answerable. If your volume is modest and your tasks vary, which describes most firms, the managed model keeps winning and you have spent nothing on infrastructure to find that out. If one high-volume job turns out to dominate your usage, you now have the evidence to price a self-hosted option properly instead of on a hunch.
The firms that struggle are the ones that treat this as a technology project rather than a process one. The tool is the easy part. The policy, the human review step, and the discipline about what data goes where are what keep you safe and out of trouble with the ATO. Get those right on a managed model first, and the question of open versus in-house mostly answers itself.
The partners who get the most out of AI are not the ones who agonise over open versus in-house. They are the ones who pick a managed tool, write down a sensible policy, train their staff to follow it, and review the results each quarter. The technology choice is reversible and cheap to change later. The trust you keep with clients by handling their financial data carefully is neither, so spend your attention there first and treat the model as the smaller decision it actually is.
Want a usage policy and tool setup built for your firm? Book a free brainstorm with us.
