Blog

Claude vs GPT-6 Sol: Reading OpenAI's Own Benchmarks

October 2026 · 7 min read · AI Strategy

Hand-drawn bar chart on a clipboard with a dashed hidden extension on the Claude-coloured bar, inspected by a magnifying glass
← Back to all posts

A launch table is the most persuasive page a model vendor publishes, and the least neutral. On 22 September 2026 OpenAI released GPT-6 Sol and GPT-6 Luna, two cheaper and faster siblings to GPT-6 Astra, with API prices cut by 50% and a set of comparisons pointed directly at Claude. If you run procurement, architecture or an AI programme at an Australian organisation, someone has probably already forwarded you that table with a one-line question: should we be looking at this?

The answer is yes, you should look. But look at the whole table, including the footnote OpenAI wrote about its own numbers, and then test on your own work before anything changes. This post walks through what the launch claims, what the asterisk says, and how to run a comparison that tells you something real about Claude and GPT-6 Sol for your workload.

What do OpenAI's GPT-6 Sol benchmarks claim against Claude?

OpenAI's launch post claims GPT-6 Sol beats or matches Claude on three benchmarks at a much lower price. On AutomationBench, Sol at high effort scores 33.2% at US$0.27 per task, against Claude Opus 5 at max effort on 26.9% for roughly eleven times the cost per task. On DeepSWE, a long-horizon coding test, Sol lands within about a point of Claude Fable 5's best score at roughly 80% lower cost. On OSWorld, a computer-use test, Sol ties Claude Opus 5 at a fraction of the price. All three results come from OpenAI's own runs.

Those are the claims as OpenAI states them in its GPT-6 Sol and Luna announcement. None of them has yet been reproduced by an independent party, and the table below keeps the source of each number visible.

GPT-6 Sol vs Claude: OpenAI's published launch claims, September 2026
BenchmarkWhat it testsOpenAI's claimWho ran it
AutomationBenchAutomation tasksSol 33.2% at US$0.27/task vs Opus 5 26.9% at ~11x costOpenAI
DeepSWELong-horizon codingSol within ~1 point of Fable 5's best at ~80% lower costOpenAI
OSWorldComputer useSol ties Opus 5 at a fraction of the priceOpenAI
API pricingList price changeSol and Luna prices cut 50%OpenAI

The footnote OpenAI put on its own table

The most useful line in the launch is one most summaries skipped. OpenAI flags that the Claude Fable 5.1 AutomationBench result understates Fable 5.1's real cost, because the figure leaves out the cost of Opus 5 fallbacks. Those fallbacks happened on around 40% of tasks in that run.

That matters for two reasons. First, it shows how much a cost-per-task number depends on the harness around the model. A setup that hands hard tasks to a larger model will look cheap or expensive depending on whether you count the hand-off. Second, it is a candid signal from the vendor itself that the comparison is partial. When the party publishing the table tells you one of its cells is incomplete, treat every cell as a starting hypothesis rather than a settled price.

We made a similar point about Claude's own pricing in our look at what a Claude Code task really costs on Opus 5.5: the number of turns, retries and cache hits per task moves the bill more than the headline token price does. The same logic applies to any model OpenAI or Anthropic puts on a slide.

Why a vendor-run comparison needs a second look

None of this means OpenAI's numbers are wrong. It means they were produced under conditions OpenAI chose, which is normal for a launch and different from your production setup. Before a benchmark row changes a procurement decision, check these variables:

  • Effort setting. Sol was run at high effort and Opus 5 at max effort. Those settings trade cost against accuracy differently on each platform, so the pairing itself is a choice.

  • Harness and tools. The scaffolding, tool definitions and retry rules around a model can move scores by several points.

  • Fallback routing. As the footnote shows, whether a run escalates to a bigger model, and whether that cost is counted, changes cost per task.

  • Task mix. A benchmark's tasks may look nothing like your invoices, contracts, tickets or code.

  • Success criteria. A benchmark pass is not the same as output your team would accept without rework.

  • Price date. List prices change. A 50% cut is real today and says nothing about next quarter.

Our earlier piece on open-weight benchmark claims versus independent reality covers the same filter for smaller vendors. It applies just as much to frontier labs, Anthropic included.

How to test Claude against GPT-6 Sol on your own work

A useful comparison takes a week or two, not a quarter. The aim is cost per accepted result on tasks you actually run, not a leaderboard score.

  • Pick 50 to 200 real tasks from the last month, with the answers your team accepted. Strip or mask personal information first so the test fits your Privacy Act obligations.

  • Run both models through the same harness, with the same tools, prompts tuned fairly for each, and the same retry policy.

  • Log every call, including retries and any fallback to a larger model, so cost per task is complete.

  • Have a reviewer mark each output as accepted, needs a fix, or rejected, without knowing which model produced it.

  • Divide total spend by accepted outputs. That figure, plus the review time, is your real comparison.

An illustrative worked example

The figures here are illustrative only. Say a Sydney finance team runs 2,000 document-processing tasks a month. Model A costs A$0.40 a task and Model B costs A$4.50, so the API bill is $800 against $9,000. On price alone, Model A wins easily.

Now add review. If Model A's output needs a fix 30% of the time and each fix takes six minutes of a staff member's time at $90 an hour, that is 600 fixes and 60 hours, or $5,400 a month. If Model B needs a fix 8% of the time, that is 160 fixes, 16 hours and $1,440. Totals: about $6,200 for Model A and about $10,440 for Model B. Model A still wins here, but by a far smaller margin than the API bill suggested, and a different error rate or task mix could flip the result. You only find out by measuring. Our ROI calculator does this arithmetic for your own volumes.

What not to conclude from the GPT-6 Sol launch

  • Do not conclude that Claude is now overpriced across the board. The comparisons cover three benchmarks, chosen by the vendor publishing them.

  • Do not conclude that the numbers are marketing fiction either. Price cuts of 50% are real and worth testing against.

  • Do not switch a production workload on a launch table. Migration costs, prompt rework and staff retraining rarely appear in cost-per-task figures.

  • Do not skip governance. APRA-regulated entities and other Australian organisations with vendor risk processes will need the change documented regardless of which model wins.

For context on how GPT-6's flagship compares, see our earlier write-up on Claude Fable 5.1 versus GPT-6 Astra, and our guide to checking AI vendor claims before you trust them.

Where this leaves a Claude decision

Cheaper competitors are good for every buyer, Claude customers included. A credible GPT-6 Sol gives you a reason to measure what your Claude workloads actually cost per accepted result, tune effort settings and caching, and check whether a smaller Claude model would do the job. Often that review finds savings without changing vendor at all.

If you want help building a side-by-side test on your own tasks and data, book a brainstorm with our team. We will help you design the comparison that answers your question, not the vendor's. You can also see how we run these engagements on our services page.

FAQ

Frequently asked questions

Is GPT-6 Sol cheaper than Claude?

On OpenAI's own launch figures, GPT-6 Sol costs far less per task than Claude Opus 5 on AutomationBench. Real cost depends on your task mix, retries, fallbacks and review time, so measure cost per accepted result on your own work.

Does GPT-6 Sol beat Claude on coding?

OpenAI says Sol comes within about a point of Claude Fable 5's best DeepSWE score at roughly 80% lower cost. That is close to parity rather than a win, and it comes from OpenAI's own run, not an independent test.

What is the asterisk on OpenAI's GPT-6 Sol benchmark table?

OpenAI notes that the Claude Fable 5.1 AutomationBench cost understates the real figure, because it excludes Opus 5 fallbacks that occurred on around 40% of tasks. It shows how much harness choices shape cost per task.

Should we switch from Claude to GPT-6 Sol?

Not on a launch table alone. Run a blind side-by-side test on 50 to 200 of your real tasks, count every call including fallbacks, add staff review time, and factor in migration and governance costs before deciding.

Ready to move from AI pilot to production?

We help mid-market Australian businesses deploy AI automations that actually reach production and deliver measurable ROI.