Blog

Sparse Experts Pay Off Unevenly: Test by Workload

November 2026 · 6 min read · Technical

Hand-drawn grid of twelve expert circles with one terracotta circle active, beside two uneven result bars
← Back to all posts

Line up the open-weight models released in the first week of October 2026 and a pattern stands out. Aleph Alpha's Kolibri-1 has 78.1 billion total parameters, 3.46 billion active per token, and 384 experts across 50 layers. Bilibili's Index-Translate has 35 billion total and 3 billion active. Cloudflare's Clef is 27 billion parameters built for cheap, fast inference. Headline sizes keep climbing while the parameters doing the work on any single request keep shrinking.

Kolibri-1 and Index-Translate both use sparse mixture-of-experts, the design now showing up across new open releases. The part that gets less attention is that its efficiency gain does not land evenly. A sparse model can be fast and capable on one kind of task and noticeably weaker on another, even when its total parameter count is larger than the dense model it is losing to.

Why are new open models sparse mixture-of-experts?

New open models use sparse mixture-of-experts because it lets a lab ship a model with a large total parameter count while keeping the cost of each request close to that of a much smaller model. The parameters are split into specialised sub-networks, called experts, and a router picks a small number of them for each token. Inference cost and latency then follow the active parameter count, while the total count still provides breadth. If you want the full mechanics, our mixture-of-experts explainer walks through them.

Kolibri-1 is a clear example. With 384 experts across 50 layers, most of the model sits idle on any given request and only a small, relevant slice computes. On paper that is the best of both worlds.

The catch the spec sheet does not show

Kolibri-1's own benchmark disclosure admits it trails dense competitors such as Qwen3.8 on general knowledge and multi-turn function calling, despite having a much larger total parameter count. The working theory is that sparse activation can under-use the model's breadth on tasks that need wide, cross-domain reasoning rather than one narrow specialist skill. The router sends each token to a few experts; when a task needs many kinds of knowledge at once, that narrow path can leave useful capacity unused.

The practical consequence is simple. For a sparse model, the total parameter count is a poor guide to how it will perform on your work. Two workloads of the same size can produce very different results on the same model.

How sparse MoE gains tend to vary by workload type, as at November 2026
Workload typeSparse MoE tends toWhat to check
Narrow specialist task, such as translation or classificationPerform well at low costAccuracy on your own examples
Broad general knowledge questionsTrail similar dense modelsHead-to-head test against a dense option
Multi-turn tool or function callingShow weaker consistencyFull conversation traces, not single turns
Drafting and judgment workLag frontier modelsSide-by-side review with a reasoning model

What this means if you plan to self-host

For an Australian team weighing self-hosting against paying API rates, the architecture changes several things at once:

  • GPU memory still has to hold every expert, so VRAM needs follow the total parameter count even though compute follows the active count.

  • Inference on a single GPU setup can be far faster than a dense model of the same total size, because far fewer weights are touched per token.

  • Fine-tuning gets harder, not easier. Adjust the wrong experts and you can destabilise routing across tasks you never meant to touch.

  • Benchmark results travel less well. Because performance varies more by workload type than with dense models, vendor numbers are a weaker predictor of your results.

Sparse activation also does not make large models cheap to run in production. We have seen Australian teams budget $4,000 to $9,000 a month for a mid-tier self-hosted setup of a 70-plus billion parameter MoE model and still hit capacity limits under real load. The memory side of that bill is covered in our look at sparse architecture and GPU costs.

How to test a sparse model against your own work

The fix for uneven performance is not to avoid sparse models. It is to test them in a way that exposes the unevenness before you commit.

Split your test set by workload type

Do not average a single score across everything. Group your test cases into narrow tasks, broad knowledge questions and multi-step tool use, and score each group separately. A model that wins overall can still fail the one category your process depends on.

Include a dense comparison

Run the same cases through a dense model of similar running cost. If the sparse model only wins on narrow tasks, you have learned where it belongs in your stack, not that it is the better model.

Test whole conversations, not single prompts

Multi-turn function calling is where Kolibri-1 admits weakness, so evaluate full traces with several tool calls in a row. Single-turn tests will flatter a model that loses the thread on turn four.

Where this leaves your model choice

Sparse mixture-of-experts is a genuine efficiency gain, not a gimmick. It is also a gain that shows up unevenly depending on what you ask the model to do. For narrow, specialist jobs, a sparse open model can be an excellent and inexpensive fit. For broad reasoning, drafting and judgment work, a frontier model such as Claude through the API will usually outperform a sparse open model of similar size on exactly the tasks where breadth matters. Headline leaderboard scores hide this, as our note on reading open-weight leaderboards explains.

If you want to work out which of your workloads suit a sparse open model and which do not, try our ROI calculator for a first estimate, or talk to us about a structured benchmark on your own data.

FAQ

Frequently asked questions

Does a larger total parameter count mean a better sparse model?

Not necessarily. Kolibri-1 has 78.1 billion total parameters yet its own disclosure shows it trailing dense competitors such as Qwen3.8 on general knowledge and multi-turn function calling.

Why is a sparse MoE model cheaper to run per request?

Only a few experts compute for each token, so inference cost and latency follow the active parameter count. Kolibri-1 activates about 3.46 billion of its 78.1 billion parameters per token.

Do sparse models need less GPU memory?

No. Every expert must be held in memory, so VRAM follows the total parameter count. The saving is in compute per token, not in the memory you need to provision.

How should we benchmark a sparse open model?

Split your test set by workload type, score each group separately, include a dense model of similar running cost for comparison, and test full multi-turn conversations rather than single prompts.

Ready to move from AI pilot to production?

We help mid-market Australian businesses deploy AI automations that actually reach production and deliver measurable ROI.