Three ways to make an AI model better at your business's specific work: write a sharper prompt, hand it your documents through retrieval (RAG), or fine-tune the model on your own examples. Vendors pitch all three like a menu. In practice they sit on a triangle, and moving toward one corner costs you something at the other two.
What each corner actually buys you
Prompting is the cheapest and fastest to change. You write clearer instructions, add examples in the prompt itself, and ship it that afternoon. The ceiling is real, though: a prompt can only carry so much context before the model starts ignoring parts of it, and you are paying for those tokens on every single call.
Retrieval-augmented generation (RAG) pulls the right passage from your own documents at query time and feeds it to the model alongside the question. It scales further than prompting because your knowledge base can be gigabytes deep while only the relevant slice gets sent each time. The cost shows up in build effort: a vector index, a retrieval step, and someone maintaining both as your documents change.
Fine-tuning changes the model's weights on your examples. It is the most expensive corner to set up and the hardest to unwind if the business changes, but it is the only one that reliably changes tone, format and judgment calls baked in through hundreds of examples rather than a paragraph of instructions.
A worked comparison for an Australian services firm
A 20-person Melbourne engineering consultancy wanted consistent scope-of-work drafting. Prompting alone got them 70% of the way for near-zero setup cost. Adding RAG over their past 200 scopes lifted accuracy further, for a one-off build of roughly $6,000 to $9,000 and a small ongoing token cost per query. Fine-tuning was quoted separately at $18,000 plus retraining every time their standard terms changed -- and for a document that needed judgment more than a fixed style, it was the wrong tool.
Prompting: cheapest, fastest, best for tasks where instructions genuinely capture the logic.
RAG: mid-cost, scales with your document library, best when the answer depends on facts that change.
Fine-tuning: most expensive and slowest to update, best when you need a consistent judgment call baked in, not just facts recalled.
Picking a corner without guessing
Start by writing down what's actually going wrong with the current output. If the model doesn't know something (a policy, a price list, a past decision) that's a retrieval gap, not a prompting or fine-tuning problem. If it knows the facts but keeps making the wrong call on tone or format, prompting usually fixes it before you touch training data at all.
Most Australian small and mid-sized businesses never need the fine-tuning corner. Prompting plus a well-built RAG layer covers the large majority of document-heavy and support-heavy workloads at a fraction of the cost and maintenance burden. We'd rather tell a Sydney client to save the $18,000 than sell them a fine-tuning project they don't need.
Combining corners instead of picking one
In practice the strongest setups don't pick a single corner, they layer two. A well-written prompt sets the tone and format, RAG supplies the facts that change over time, and fine-tuning stays in reserve for the rare case where dozens of prompt iterations still can't get the judgment call right. Treating this as a single either/or decision is where most businesses overspend -- they either under-invest in RAG and keep patching prompts that were never going to hold a growing document set, or they jump straight to fine-tuning because a vendor pitched it as the 'proper' solution.
A useful discipline before committing budget to any corner: write down three real examples of the model getting something wrong, and diagnose each one separately. If the fix is 'it didn't know that,' that's retrieval. If the fix is 'it knew it but phrased it wrong,' that's prompting. Only when you've genuinely ruled out both, and you're seeing a consistent, hard-to-articulate judgment gap across dozens of examples, does fine-tuning start to earn its higher cost and lower flexibility.
A note on switching later
None of these choices are permanent. A business that starts with prompting alone can add RAG six months later without throwing away the original work, and the reverse holds too -- a RAG layer built now becomes the training data source if fine-tuning ever becomes genuinely necessary. Treat the first choice as a starting point tuned to today's budget and today's problem, not a five-year architectural commitment. The Melbourne consultancy in the example above started with prompting only, added RAG four months in once their scope library hit 200 documents, and still hasn't needed fine-tuning eighteen months later.
If you're weighing this trade-off for a live project, we can walk through your specific workload and put a number on each corner before you commit budget. Get in touch through /contact and bring the failure cases you've actually seen, not the hypothetical ones.
