Most of the open-weight coverage in 2026 has chased size. Each new 400B-class release gets the benchmark charts and the headlines. The shift that matters more to a typical Australian business has gone the other way: reasoning models at 3B, 8B and 30B parameters that run on a single GPU or a strong laptop, and that now ship with reasoning built in.
IBM's Granite 4.2 line and NVIDIA's Nemotron Nano variants, both released around September 2026, are the clearest recent examples. A year ago, getting a small model to reason meant bolting chain-of-thought prompting on top and hoping. Native reasoning at these sizes changes one specific thing: which workloads can now stay entirely inside your own building.
Why do small reasoning models matter for Australian businesses?
Small reasoning models matter because they make on-premises AI practical for narrow tasks that could not previously leave the building. For parts of aged care, health and financial services, the Privacy Act and client contracts make sending data to an offshore cloud a real constraint. A capable 8B model on local hardware can classify, extract and check documents without any record leaving your network.
That is a different argument from cost. We have covered the cost case for routing narrow tasks to 8B models separately. The point here is deployability: a whole class of sensitive, well-defined work that was previously a choice between a cloud API and doing it by hand now has a third option.
The practical benefits line up like this:
Data residency: the model, the inputs and the outputs all stay on hardware you control.
Latency: a local 8B model on decent hardware often responds faster than a round trip to a large cloud-hosted model, which helps interactive internal tools.
Running cost: a single mid-range GPU can serve a small reasoning model continuously, often for under $10,000 a year in hardware and power at modest volume.
That last figure covers hardware and power only. It leaves out the staff time to run, patch and monitor the model, which is usually the bigger number. Our Granite 4.2 cost breakdown shows how the full figure grows once people are counted.
Where small models still fall short
A small reasoning model is not a cheaper version of a frontier model. The gaps are predictable, and they decide which work you hand it:
Long, multi-step agentic tasks, where a larger model's broader training still wins on reliability.
Edge cases and unusual phrasing, where a narrower training distribution shows through more often.
Tool use across many different APIs, where large frontier models still hold a clear accuracy edge in our client testing.
| Workload | Data must stay on-site? | Better fit |
|---|---|---|
| Classifying incoming documents | Yes | Small reasoning model, local |
| Extracting fields from forms | Yes | Small reasoning model, local |
| Fixed set of internal tools | Yes | Small reasoning model, local |
| Multi-step client work across systems | No | Claude |
| Complex coding | No | Claude |
| Agentic workflows spanning several systems | No | Claude |
The table is the decision in short form. If a task is narrow, well defined and bound by residency, a small local model is worth a proper evaluation. If it is broad, multi-step or client-facing, the larger models, Claude included, still earn their keep.
A worked example: an aged care provider
Take an illustrative aged care provider in regional New South Wales that receives several hundred care notes a week and wants them classified by risk level, with a few structured fields pulled out. The notes contain health information, and the provider's board has decided they will not leave its own systems.
This is close to the ideal small-model workload. The task is narrow, the categories are fixed, and the inputs are short. A Granite or Nemotron Nano model on one local GPU can handle the volume. The illustrative budget is around $8,000 a year in hardware and power, plus perhaps $15,000 of initial setup and testing, plus ongoing staff time to maintain it. Without a residency rule, a cloud API might well be simpler. With the rule, the alternative is a person reading every note.
What the provider should not do is stretch the same model into drafting family correspondence or coordinating across rostering, billing and clinical systems. Those are the multi-step, edge-case-heavy tasks where small models slip, and where a managed model with proper contractual controls is the better tool. Sites with no reliable connection face a sharper version of the same trade-off, covered in our piece on on-device models for remote mining sites.
Choosing between Granite and Nemotron Nano
Both families target the same niche, so the choice comes down to your own test results and your runtime. Run each on a fixed sample of your real documents, check the licence on the model card, and confirm your serving stack supports the model well. Our guide to Ollama, vLLM and LocalAI covers the runtime side, and our Nemotron 3 review explains why the larger Nemotron model is a watch-list item while Nano is a fair shortlist candidate.
What not to conclude
The trend is not that small models have caught up with large ones. They have become reliable enough for narrow work at sizes that fit existing hardware. The question to track through 2026 is not which lab ships the biggest model but how small a dependable reasoning model can get, because that is what changes what an ordinary business can run on its own premises.
If you have a residency-bound workload and want to know whether a small local model fits, start with our AI readiness assessment or talk to us.



