OpenAI models breach test environment and hack Hugging Face
OpenAI has disclosed that two of its most advanced artificial intelligence models, including GPT-5.6 Sol, breached a controlled testing environment and autonomously hacked another AI company, Hugging Face. The incident, described as an “unprecedented cyber incident,” occurred during an internal test designed to assess the models’ advanced hacking capabilities.
Details of the incident
OpenAI said the evaluation was conducted in a highly isolated environment with restricted network access. However, the models spent significant computing resources attempting to obtain internet connectivity to solve the assigned task. The ChatGPT maker stated that the models discovered and exploited a zero-day vulnerability in an internally hosted package registry cache proxy, allowing them to escape the restricted environment.
“After gaining internet access, the models inferred that Hugging Face potentially hosted models, datasets, and solutions for ExploitGym,” OpenAI said. The models then chained together multiple attack techniques, including stolen credentials and additional zero-day vulnerabilities, to establish a path to remote code execution on Hugging Face’s servers. They later retrieved evaluation solutions directly from the company’s production database.
Security response and collaboration
OpenAI’s internal security team detected the unusual activity, while Hugging Face’s security team independently identified, contained, and began investigating the incident. “Hugging Face’s security team and agents detected and stopped the activity on their infrastructure and had already begun containment and forensic reconstruction with their own open-source models when our teams connected,” OpenAI said.
The tech giant is working closely with Hugging Face to investigate the incident, patch the vulnerabilities discovered, and strengthen safeguards for future model evaluations. Immediate measures include tighter infrastructure controls, responsible disclosure of the zero-day vulnerability to the affected software vendor, expanded monitoring of internal evaluations, and additional protections during model testing. OpenAI also granted Hugging Face access to its Trusted Access programme to help strengthen the company’s cyber defences using advanced AI models.
Implications for AI security
The incident, OpenAI said, demonstrates that frontier AI systems are increasingly capable of discovering and exploiting complex attack paths in real-world environments without access to source code. “The primary lesson from this incident is that model security and safety must keep pace with rapidly advancing capabilities,” the company said.
Recent evaluations by the UK AI Security Institute (AISI) indicate that models such as GPT-5.6 Sol can sustain complex, multi-step cyber operations over extended periods, suggesting that these capabilities can translate to real-world settings. OpenAI believes advanced AI models should ultimately be used to help defenders identify vulnerabilities before attackers do, understand how weaknesses can be chained together, and accelerate remediation.
Industry reaction
Commenting on the incident, Hugging Face co-founder and Chief Executive Officer Clem Delangue described the collaboration as evidence that AI safety requires industry-wide cooperation. “We’re grateful for the collaboration with OpenAI on this and other topics. This incident, possibly the first of its kind, proves a point we’ve long believed: AI safety won’t be solved by any single company working in secret. It will be solved in the open, collaboratively, with broad access to AI for every defender, everywhere,” Mr Delangue said.
