The one-million-token context window in MiniMax M3, released in June 2026, runs on a technique the company calls MiniMax Sparse Attention. Long context windows are turning up in most new models, and the engineering underneath them shapes cost, speed, and reliability for anyone building on them. This is a plain-language guide for Australian technical teams who need to decide how much context they actually require, and what the bigger window really costs.
Why attention is the bottleneck
A language model uses attention to relate each token in a prompt to the others. In the standard form, that work grows with the square of the prompt length, so doubling the input can roughly quadruple the compute. That maths is why early models capped context at a few thousand tokens. To reach a million tokens without the cost climbing out of reach, modern models use sparse attention, which computes only the most relevant relationships between tokens rather than every possible pair.
The saving is large, but it comes from approximation. Instead of reading every connection in full, the model decides which links are likely to matter and skips the rest. For most prompts that choice is sound. The risk shows up when the single detail that matters is buried in a part of the prompt the model chose to skim.
Sparse attention makes long context affordable, but it approximates rather than reading every token pair in full.
Answer quality can dip when a critical detail sits deep inside a very long prompt.
More context is not automatically better, because the signal you need can be diluted by surrounding noise.
What sparse attention changes about cost
The headline benefit of sparse attention is that a million-token window stops being prohibitively expensive to compute. The catch for a business is that affordable is not the same as cheap. You still pay for every token you send, and a habit of pasting whole document libraries into the window adds up fast.
Cost scales with how much you put into the window, not only the answer you get back.
A one-million-token prompt is usually slower and dearer than a focused ten-thousand-token one.
The compute saving from sparse attention does not cancel the per-token price of a bloated prompt.
This is where teams get caught. The model can technically accept a million tokens, so it feels free to use them all. The invoice tells a different story at the end of the month, and by then the habit is baked into the system and harder to unpick.
Designing systems around context, not against it
For most Australian business systems, the practical lesson is that a giant context window is a tool for specific jobs rather than a default setting. Sending an entire document set on every request is slow, expensive, and can lower answer quality. A more disciplined pattern serves most workloads better:
Use retrieval to find and send only the relevant passages for routine queries.
Reserve the full context window for tasks that genuinely need a whole document at once.
Test answer quality at several context sizes rather than assuming bigger is better.
We design client systems on Claude, from Anthropic, using exactly this discipline. Claude offers a large context window when a task needs it, and we pair it with retrieval so the common case stays small, fast, and inexpensive. Open-weight long-context models such as MiniMax M3 slot into the same design where data has to stay onshore, and the same rule applies regardless of which model sits underneath.
A worked cost and reliability check
Consider a Sydney team processing a few thousand documents a month. If every request pushes a large prompt through the model, the monthly input bill can sit around $5,000. Switching to a retrieval design that sends only the relevant clauses can bring the same job down toward $1,500, with no drop in answer quality. The work is identical. The only change is how much you send per request.
Reliability follows the same logic. Smaller, focused prompts are easier to test, easier to debug, and less likely to surprise you in production than a one-million-token prompt whose behaviour is hard to predict. When something breaks at 2am, a tight prompt is far quicker to reason about than a sprawling one. For a regulated workload that touches personal data under the Privacy Act, that predictability is worth real money.
How to size context for your own workload
Before committing to a long-context design, a short checklist keeps the decision grounded in your work rather than the model spec sheet:
Measure the real token size of a typical request, not the maximum the model allows.
Compare answer quality from a full-context prompt against a retrieval-based one on your own data.
Price both patterns at your actual monthly volume before rolling either out widely.
Check whether any of the data must stay on Australian infrastructure, which may point to a self-hosted open-weight model.
A Melbourne firm that runs this comparison once usually finds the retrieval path wins on both cost and reliability for the bulk of its queries, with the full window reserved for the genuine whole-document cases. A short design review of this kind, costing around $3,500, tends to save many times that across a year of running prompts.
The takeaway
Sparse attention is what makes a million-token context window possible, and it is a real advance in model engineering. It is not a reason to send everything, every time. The Australian teams that build well treat context size as a deliberate design decision tied to the task and the budget, then measure the bill as they go. Book a brainstorm if you want help designing a context strategy for your own system.
