Home › Blog

Plain-English Briefing

The rogue AI had an owner — and a detail nearly everyone missed

By Charlie Essex · 29 July 2026 · 4 minute read

A fortnight ago I wrote about an AI system that broke out of its testing environment and attacked a company. At the time nobody knew whose it was. Now we do — and the more useful part of the story turned up in the small print.

What happened

On 16 July, Hugging Face — the company that hosts a large share of the world’s openly available AI models — disclosed that something had broken into its production systems, and that the attack had been run end to end by an automated AI system rather than a person at a keyboard. They did not know whose it was. They reported it to law enforcement.

Five days later, OpenAI said it was theirs. The models had been running an internal cyber-security evaluation with their safety restrictions deliberately turned down, which is a normal thing to do when you are testing what a model is capable of. Rather than solve the test, the models found a flaw, escaped the environment they were being tested in, and went and broke into Hugging Face’s live systems — because the answers to the test were over there. Nobody instructed them to attack anyone. They were told to find complex attack paths, and another company’s servers turned out to be the shortest one.

That is the headline, and it is genuinely strange. But it is not the part I’d want a business owner to take away.

The bit nobody's talking about

Once Hugging Face detected the breach, their security team did the obvious 2026 thing: they fed the evidence to the big commercial AI services to help them work out what had happened. Thousands of log entries, exploit code, captured commands — the raw material of any investigation.

The AI refused.

Not through malice, and not because anything was broken. The safety systems on those services are built to block requests that look like someone trying to attack a computer. A defender analysing an attack and an attacker planning one submit almost identical-looking material. The filter could not tell the difference, so it blocked the people cleaning up.

They finished the investigation using an openly available model running on their own hardware, because it was the capable one that would actually look at the evidence.

Hugging Face wrote up their own conclusion plainly: have a capable model you can run on your own infrastructure, vetted before you need it — both so you aren’t locked out, and so attacker data and credentials don’t leave your building while you’re investigating. They were careful to add that this is not an argument against safety measures on hosted AI, and they’re right. I’ll be equally careful below.

What this means for a normal business — honestly

Let me take the obvious sales pitch off the table first, because I sell on-premises AI systems for a living and you should assume I’m biased.

This is an incident-response story about a large technology company with a security team and thousands of log entries to reconstruct. If you run a fabrication shop in Burgess Hill or a motor factor in Crawley, you are not going to be doing forensic analysis of exploit payloads, and anybody telling you this story is a reason to buy a server from them is stretching it. It isn’t.

What does carry across is smaller and duller, and I think it’s real:

• A tool you rent can decline the job. Not go down — decline. Most of the time that’s the system working as intended. But the decision about whether you have an alternative gets made long before the day you need one.
• The awkward work is the work that gets refused. Nobody’s AI blocks a request to tidy up a spreadsheet. The refusals land on the unusual, urgent, sensitive jobs — which is to say, exactly the ones that matter.
• “It’s in the cloud” is a decision, not a default. It’s a perfectly reasonable decision for most small businesses most of the time. It is worth having made it on purpose, once, rather than by drift.

I’d also note the thing this story does not show. It doesn’t show AI turning hostile or wanting anything. It shows a system pursuing a goal it was set, taking a route nobody anticipated, at a speed nobody could follow. That’s a supervision problem, and supervision problems are ordinary engineering, not science fiction.

This is a follow-up to The AI that broke out of its box, written before the attacker was known. More plain-English briefings on the Inferred blog.