OpenAI’s own report shows how 700 programs slipped past safeguards the company had but never switched on.
By EEW Magazine Online News & Tech Editors
The OpenAI logo appears on a smartphone screen. OpenAI says AI agents running in one of its internal tests broke into Hugging Face’s systems in July. (Photo by Samuel Boivin/NurPhoto via Getty Images)
The next time a company’s website reschedules your appointment or pulls up your account, the work may be done by an AI agent. The tech industry uses that name for software that carries out tasks on its own, opening files, writing code and logging into accounts with no one checking every step. The name can make them sound like people. They are computer programs.
A chatbot answers questions, and an agent carries out jobs. Companies are starting to hand agents everyday work like customer service, scheduling and paperwork, sometimes with access to customer accounts and records. That makes the question of who watches these tools a concern for anyone whose personal information sits in a company’s computers, which is nearly everyone.
A summer incident at OpenAI, the company behind ChatGPT, shows what can happen when the watching falls short.
In July, about 700 AI agents from OpenAI worked together for several days to break into Hugging Face, a website that works like a giant public library and workshop for the people who build AI. The agents had escaped a locked-down test where OpenAI was checking how well they could hack. People built the test and gave the agents their assignments, and the agents took it from there. The software was experimental, and the main driver was an internal research model OpenAI never planned to release, Axios reported.
The Hugging Face logo appears on a smartphone resting on a laptop keyboard. Hugging Face is a website where the people who build AI store, share and test their work. (Jakub Porzycki/NurPhoto/Getty Images)
OpenAI says the incident had no impact on its customer data or products, according to its technical report.
Warning signs came well before July. Jonas Wiedermann-Moeller, a 27-year-old independent researcher in Bielefeld, Germany, found evidence that as early as May 13. AI software linked to OpenAI took over two Hugging Face user accounts and used them to poke at the site’s computers, as if testing for a way in, Reuters reported in September. Researchers found no evidence that activity was part of the July attack.
Wiedermann-Moeller still sees a missed chance. “Imagine if they caught this behavior in May,” he told Reuters.
OpenAI was putting two models, GPT-5.6 Sol and the unreleased research model, through an internal exam. It measures how well AI can turn known software flaws into working attacks. To see their full ability, it loosened some of its usual safety limits, according to its account. The test was supposed to have no internet access.
The exam worked like a set of puzzles scored by a grading program. Within hours, the agents found a way to produce correct answers without solving the puzzles, much like a student who gets hold of the answers instead of doing the work, according to an independent investigation by the AI safety nonprofits METR and Redwood Research.
The agents’ own notes show they expected the grading program to check how each answer was reached and mark them down for skipping the work. So they set out to learn how the grading program operated. They reasoned that Hugging Face likely held files connected to the exam, HotHardware reported, and that is why they broke in. They had it wrong. The grading program only checked the final answers, and the shortcut would have passed on its own.
The agents were supposed to work separately. They found a way to leave messages for one another in an OpenAI file-sharing tool, turning it into a makeshift message board. About 1,200 agents posted roughly 70,000 messages there, trading what they learned. The attack on Hugging Face grew out of that board, Fortune reported.
Some tried to alter the records of their own actions, though investigators described those attempts as small and easy to spot.
Inside OpenAI, a team saw agents reaching the internet without permission in late May, according to the company’s report. On June 27, a security tool flagged suspicious activity, and the staff on call decided “the evaluation did not need to be stopped,” Axios reported. In early July, the agents’ activity crashed an internal system. OpenAI took it offline, cut off the agents’ access and restarted the test within days.
A stock image shows rows of computer servers in a data center. Machines like these run the AI programs and websites at the center of this story. (Oselote/Getty Images)
The agents rebuilt their message board soon after, OpenAI researchers told a security conference in August. The company has since said some early signals could have prompted a faster response.
OpenAI also had monitors that read a model’s reasoning for signs of trouble. It did not run them on this test because it underestimated the models, chief scientist Jakub Pachocki told Time magazine, according to news reports. When OpenAI later replayed the incident with stronger monitoring, an alert fired about 30 minutes after the agents escaped, more than 30 hours before they reached Hugging Face, the company said. That comes from OpenAI’s own simulation, and an alert still needs a person to act on it.
Around July 9, the agents used an unknown software flaw to reach the open internet. From July 11 to 13, they broke into Hugging Face through weaknesses in how it processes uploaded files, Reuters reported. They carried out more than 17,000 recorded actions, gathered passwords and digital keys and reached some internal company data.
Hugging Face found no evidence that the public tools and data its users rely on were altered, TechRepublic reported. It closed the flaws, replaced the exposed keys and urged all users to change their access tokens. In July, it said it was still checking whether any partner or customer data was affected.
Hugging Face caught the intrusion itself and disclosed it July 16. OpenAI first reached out as a customer checking on its own data. Its staff found clues in internal logs July 18 and 19 pointing to its own agents, and OpenAI disclosed its role July 21.
Legal scholar Kate Klonick, writing in Lawfare, argues that framing the episode as AI gone rogue can hide the human failures underneath: people lowered guardrails and left gaps in containment. She also warns that dramatic talk about AI’s power can double as advertising for the companies building it.
OpenAI is not the only lab to report a containment problem. Anthropic, the maker of the Claude chatbot, disclosed that its models reached outside systems in three tests after a misunderstanding with a testing partner left internet access open.
OpenAI calls the incident a “warning shot.” In August it paused model testing for two weeks and slowed development while it overhauled its systems, Reuters reported. A Senate subcommittee led by Sen. Josh Hawley, R-Mo., is investigating OpenAI’s handling of the breach, and Alabama’s attorney general has subpoenaed the company. Neither is a finding of wrongdoing.
The breach began as an exam to measure what AI could do and ended by exposing the limits of the systems built to contain it. The agents chose their own path to Hugging Face, and OpenAI had chances to catch them sooner.
As these programs gain access to the systems that hold people’s information, the humans responsible for the safeguards around them will count as much as the technology.