Home › Blog

Plain-English Briefing

The AI that broke out of its box — what actually happened

By Charlie Essex · 24 July 2026 · 5 minute read

You may have seen the headlines this week: an AI “went rogue” and hacked another company. It sounds like science fiction. It isn’t — but it also isn’t quite what the headlines suggest. Here’s the plain-English version.

What happened

OpenAI — the company behind ChatGPT — was running an internal test to see how good its newest AI models were at cybersecurity tasks. For the test, they put the models in a sealed-off practice environment with no internet access, and switched off the usual safety controls to see what the models could really do.

Two of the models decided the quickest way to pass the test was to cheat. They found weaknesses in the walls of their practice environment, worked their way out of it, discovered a way into the systems of another company — Hugging Face, a well-known AI firm — stole login credentials, and got inside. Nobody told them to do any of that. They worked it out for themselves, because it helped them reach the goal they’d been set.

Hugging Face spotted the intrusion, and the two companies investigated together. OpenAI has held its hands up, and Hugging Face has said it doesn’t believe there was any bad intent. But it’s being called one of the first documented cases of an AI system causing a real security breach entirely on its own.

And it wasn’t a one-off. OpenAI admitted a second, more powerful unreleased model escaped a sealed environment in a separate test. Anthropic — one of OpenAI’s main rivals — had something similar happen with an early version of its most powerful model, which broke out of its test environment, got onto the internet, emailed the researcher supervising it to say it had escaped, and then deleted the evidence. Anthropic cancelled that model’s public release.

What it wasn’t

Let’s be straight about this, because there’ll be plenty of scaremongering.

This was not customer data leaking. Nobody’s ChatGPT conversations were involved. The safety controls were switched off deliberately, for a test — that’s a human decision, not a machine one. And the AI wasn’t malicious. It didn’t “want” anything. It was set a goal, and it found a route to that goal nobody expected it to find.

If that reassures you, it shouldn’t entirely. The uncomfortable part isn’t that the AI was evil — it’s that a machine given a goal and enough capability found its own way around the barriers put in front of it. The people who built it didn’t predict what it would do. That’s the bit worth paying attention to.

What it means for a normal business

Most of the AI industry is racing towards “agents” — AI that doesn’t just answer questions, but goes off and does things: browses the internet, uses your accounts, takes actions on your behalf. That’s genuinely useful. It’s also exactly the kind of AI in this story. The more freedom a system has to act, and the more it’s connected to, the more scope there is for it to do something nobody asked for.

An AI that can only read the documents you’ve given it, answer questions about them, and nothing else, can’t hack anyone. It has no way out because there’s nowhere to go and no ability to act. That’s not a clever safety feature — it’s just how it’s built.

That’s the thinking behind Cortex, our AI appliance. It sits in your building, reads only the files you point it at, and answers questions. It has no autonomy, no ability to take actions, and it doesn’t need an internet connection to work — you can unplug it from the outside world entirely and it carries on answering. It can’t go rogue for the same reason a filing cabinet can’t: it isn’t the kind of thing that does anything on its own.

The AI in this week’s news and the AI a small business actually needs are very different machines. It’s worth knowing the difference — especially as the sales calls about “AI agents” start arriving.