Blog

Claude Users and the Hugging Face Incident: What to Ask

September 2026 · 6 min read · Technical

Notebook sketch of packages on a conveyor being inspected before passing a shield
← Back to all posts

In August 2026 OpenAI published its findings on a security incident involving Hugging Face, along with the changes it is making to model security, monitoring and alignment work. The useful thing for an Australian business is not the incident itself. It is the category of risk the incident sits in, because most AI due-diligence checklists, as at 2026, still do not cover it.

If you run Claude through the API or through Claude Code, this is not a story about your provider being breached. It is a story about the part of the AI supply chain that sits underneath almost everyone, and the questions you ask about it are different from the ones you ask a vendor.

A model hub is a distribution channel, not a vendor

Hugging Face is where open-weight models live. When a team downloads a set of weights, a tokeniser config or an inference script from a public hub, they are pulling executable artefacts from a third party into their own environment. That is the same shape as pulling an npm package or a PyPI wheel, and it deserves the same suspicion.

Standard AI vendor questionnaires do not treat it that way. They ask about data residency, retention, subprocessors and model training. All reasonable questions. None of them touch the case where the artefact you downloaded is not the artefact the publisher intended you to have.

The distinction matters because the two risks land in different places on your risk register:

  • Vendor risk sits with the provider. If you use Claude through the API, the weights never leave Anthropic's infrastructure and you inherit their controls, for better or worse.

  • Supply chain risk sits with you. If you download weights and run them on your own GPUs, you own the integrity of that artefact from the moment it lands on your disk.

  • Most Australian mid-market teams carry both, because the hosted model does the customer-facing work while someone in engineering is quietly experimenting with an open-weight model on a workstation.

The second and third of those are the ones that go unmonitored. We have walked into engagements in Sydney and Melbourne where the formal AI vendor register listed two providers, and the actual model footprint, once we looked at what was installed, ran to eleven.

Where the risk actually sits, by consumption mode

It helps to separate the two modes before you write a single control. The table below is the version we hand to clients at the start of a governance review.

QuestionClaude via hosted APISelf-hosted open weights
Who holds the model artefactAnthropicYou do, on your own disk
Who verifies its integrityThe provider, as part of their controlsYou, on every single pull
If the artefact is tampered withA provider incident with a contractual remedyYour incident, with no counterparty
Blast radius if compromisedBounded by the API surfaceBounded only by the tool and network access you granted it
Who you can ask for evidenceA named vendor, under contractA public registry you have no relationship with
Who owns it under CPS 234Shared and documentedEntirely you

Does the Hugging Face incident affect teams running Claude?

Not directly, if Claude is the only model you consume. When Claude is used through the API, Claude Code or Cowork, the model artefact never enters your custody, so there is nothing on your side to verify, no checksum to compare and no poisoned download to catch. Your exposure is the provider's exposure, which is exactly what a vendor questionnaire is built to assess. The incident becomes yours only when someone in your organisation also downloads open weights.

The moment someone on your team pulls open weights, that changes. You have taken custody of a binary from a public registry and you are running it with whatever tool access you gave it. If that model has file system or network reach through an agent framework, a compromised artefact is a remote code execution problem, not a content-safety problem. Those get handled by different people in most organisations, and the second team usually has not been told.

This is why we have argued in our twenty-question AI vendor due-diligence checklist that the questionnaire alone is insufficient. It assesses the front door. Model downloads come through the loading dock.

A due-diligence pass you can run this week

None of this requires a programme of work. It requires an afternoon and someone willing to look:

  • Inventory every model artefact actually running in your environment, not the ones on the vendor register. Include workstations and anything a developer spun up on a cloud GPU.

  • For each open-weight model, record where it was downloaded from, who downloaded it, whether the hash was verified against the publisher, and what tool permissions it currently holds.

  • Check whether your inference stack loads pickled formats. Prefer safetensors. A pickle file executes code on load, which is the mechanism most model supply-chain attacks depend on.

  • Decide which workloads are allowed to touch open weights at all. Customer data and anything covered by the Privacy Act is the obvious line, and for APRA-regulated entities CPS 234 already requires you to be able to answer this.

  • Put the model registry itself into your third-party register, alongside your cloud provider. It is a dependency whether or not you ever signed a contract with it.

For a fifteen-person firm this is roughly a day of work, or about $1,800 of internal time. The alternative is discovering the answer during an incident, when the same question costs a great deal more and gets asked by someone else.

If your team runs Claude in the desktop app, our note on what Cowork can and cannot access covers the equivalent boundary question for that surface.

What we will not claim

We build on Claude, and we are not going to use somebody else's incident to suggest Claude is immune to an equivalent one. It is not. Anthropic operates infrastructure, ships model updates and depends on suppliers, and any of those can fail. Anyone telling you otherwise is selling.

What we will say is narrower and more useful. Consuming a model through a hosted API removes an entire class of artefact-integrity risk from your side of the boundary, because you never take custody of the artefact. That is a real structural difference, and it is worth understanding before you decide that self-hosting an open-weight model is the conservative choice. It often is not.

The honest framing for an Australian board is that hosted and self-hosted trade one risk for another. Hosted concentrates risk in a provider you can contract with and audit. Self-hosted distributes it across a registry you have no relationship with and controls you have to build and staff yourself. Neither is safe by default. Only one of them is commonly assumed to be.

If you want a second set of eyes on where your model footprint actually sits, book a time with us and we will run the inventory pass with your team.

FAQ

Frequently asked questions

Does the Hugging Face incident affect us if we only use Claude through the API?

Not directly. When Claude is consumed through the API, Claude Code or Cowork, the model weights stay on Anthropic infrastructure and you never take custody of the artefact. The incident matters to you only if someone in your organisation is also downloading open-weight models onto your own machines, which is more common than most vendor registers show.

What is the difference between AI vendor risk and AI supply chain risk?

Vendor risk is about a provider you have a contract with: their retention, residency, subprocessors and controls. Supply chain risk is about artefacts you download and execute yourself, such as model weights or inference scripts from a public registry. Standard AI questionnaires cover the first and almost never the second.

Why does the model file format matter for security?

Pickled formats execute code when they are loaded, which makes them a direct code execution path. The safetensors format was designed to avoid this. If your inference stack still loads pickle files from public sources, that is the single highest-value thing to change.

Does APRA CPS 234 apply to models we download ourselves?

CPS 234 requires regulated entities to maintain information security capability commensurate with the threats, including for information assets managed by third parties. A model artefact pulled from a public registry and run on your infrastructure is squarely within that. If you cannot say who downloaded it and whether it was verified, you have a gap to document.

Is self-hosting an open-weight model safer than using a hosted API?

It is different, not safer by default. Self-hosting removes the provider from your data path but hands you the integrity, patching and monitoring burden for an artefact you did not build. Hosted concentrates risk in an accountable counterparty. Which trade suits you depends on your data classification and whether you have the security staff to carry the second option.

Ready to move from AI pilot to production?

We help mid-market Australian businesses deploy AI automations that actually reach production and deliver measurable ROI.