Blog

Claude vs OpenAI: Check AI Vendor Claims Before You Trust

October 2026 · AI Strategy

Hand-drawn magnifying glass held over a vendor claim card, with a dated checklist beside it
← Back to all posts

In late September 2026 OpenAI said an internal model had reportedly resolved the Navier-Stokes Millennium Prize problem, along with more than 100 other long-standing open maths problems, since late August. In the same announcement it said it was forming an independent Advisory Group on Mathematics and Artificial Intelligence to help review and communicate results like that one.

Whatever the maths turns out to show, the order of events is the useful part for anyone buying AI. A vendor made a striking capability claim, then built outside machinery to make the claim credible. For a business choosing which AI vendor to trust with customer work, that sequence is backwards, and it is worth knowing how to spot it.

What did OpenAI announce about maths and AI?

OpenAI announced on 21 September 2026 that an internal model had reportedly solved the Navier-Stokes Millennium Prize problem and more than 100 other open maths problems since late August 2026. It also formed an independent Advisory Group on Mathematics and Artificial Intelligence, hosted at the Institute for Advanced Study, to review and communicate results of this kind.

The group is staffed by mathematicians from Cambridge, Berkeley, Oxford, Stanford and Harvard. OpenAI's own announcement says it was set up partly in response to an open letter from mathematicians. That letter warned about the risk of AI labs treating open problems as a benchmark to chase rather than a field to respect.

None of this has been independently confirmed at the time of writing. Treat the Navier-Stokes result as a company-reported claim until the people now reviewing it say otherwise.

How do you verify an AI vendor's capability claim?

To verify an AI vendor's capability claim, ask who checked it, when they checked it, and whether that checking existed before the announcement. A claim reviewed by an outside group that was already running carries more weight than one reviewed by a group assembled after public pushback, because the earlier group had no launch to protect.

In practice that becomes a short set of questions you can put to any vendor, Claude's maker included:

  • Was the review process published before the model or result was announced, or after?

  • Who outside the company reviewed it, and can they publish disagreement?

  • Is the claim reproducible on your own data, or only on the vendor's chosen benchmark?

  • What happens to the claim if the outside reviewers find a flaw?

  • Is there a written policy that triggers extra safeguards when a model gets more capable?

The table below sorts the common signals into the ones that suggest review was built in and the ones that suggest it was added later.

Signals that a vendor claim was checked before launch, or after
SignalBuilt in before the claimBolted on after the claimWhat to ask
Review frameworkPublished and updated ahead of releasesAnnounced alongside or after the claimWhen was this framework first published?
Outside reviewersStanding group with a track recordNew group formed in response to criticismHow long has this group existed?
EvidenceMethod and data released for checkingHeadline result, details to followCan we see the method?
Your own testVendor supports a pilot on your dataVendor points to its benchmarkCan we run it on our cases first?
DisagreementReviewers can publish dissentReviewers help communicate resultsCan reviewers say no in public?

Where Claude's approach differs

Claude's development runs under the Responsible Scaling Policy, with capability and safeguard assessments that are published and updated ahead of new model releases. That is a standing framework, not one assembled in reaction to a news cycle. The difference matters less for maths results and more for the ordinary question a buyer is really asking: what governance was running before anyone was watching?

This is not a claim that Claude needs no checking. We have written before about outside evaluators working inside the company that builds Claude and about what OpenAI's misalignment reports tell vendor buyers. Both point the same way. The useful test is whether review is routine, not whether a vendor can produce a reviewer when asked.

What this means for Australian buyers

Most Australian businesses will never need a model that proves theorems. They will, however, sign contracts on the strength of vendor claims about accuracy, safety and data handling. Under the Privacy Act, and for APRA-regulated entities managing third-party risk, the burden of showing that a vendor was fit for purpose sits with you, not the vendor.

Here is an illustrative example. A Brisbane insurer budgets $180,000 for a claims-triage pilot on the strength of a vendor's accuracy figure. If that figure was never checked outside the vendor, the first real test happens in production, on customer files. Spending $15,000 up front on a two-week evaluation against 500 of the insurer's own past claims is cheap insurance against finding the gap later.

Our AI readiness assessment covers this step, and the vendor safety questions we suggested after GPT-Red still apply.

What not to conclude from this

  • That the maths result is false. It may well hold up; the point is that it has not yet been checked.

  • That advisory groups are a bad sign. A good one formed late is still better than none.

  • That any one vendor's governance is finished. Ask the same questions of Claude as of everyone else.

The habit worth building is simple: when a vendor makes a big claim, look for the date on the review process before you look at the result.

If you are weighing AI vendors and want a plain read on what is verifiable and what is marketing, book a short call with us. We will walk through your shortlist and the claims each vendor is making.

FAQ

Frequently asked questions

Did OpenAI solve the Navier-Stokes Millennium Prize problem?

OpenAI reported in September 2026 that an internal model had resolved it, but the result had not been independently confirmed when announced. Its new advisory group of outside mathematicians was formed partly to review claims like this one.

What is OpenAI's Advisory Group on Mathematics and AI?

It is an independent group hosted at the Institute for Advanced Study, staffed by mathematicians from Cambridge, Berkeley, Oxford, Stanford and Harvard, set up to help review and communicate AI results in mathematics.

What is Claude's Responsible Scaling Policy?

It is the standing framework the company behind Claude uses to assess model capabilities and safeguards, with assessments published and updated ahead of new model releases rather than assembled after a launch.

How should a business test an AI vendor's accuracy claim?

Run a short pilot on your own past cases before signing, compare results against the vendor's headline figure, and ask who outside the vendor reviewed the claim and when that review process started.

Ready to move from AI pilot to production?

We help mid-market Australian businesses deploy AI automations that actually reach production and deliver measurable ROI.