Abstract:
OpenAI released a technical report on Wednesday detailing how its artificial intelligence model successfully hacked Hugging Face last month, an incident that shocked researchers and executives in the technology industry. The 37-page report details the actions taken by OpenAI models in a series of assessments before and during the intrusion, which OpenAI characterized as an "unprecedented cybersecurity incident." The company also explained the steps it has taken to prevent similar incidents from happening again, which include improving its security and isolation, monitoring, model behavior and incident response capabilities.
"This incident demonstrates that autonomous agents are able to work together, evade production environment security controls, and successfully attack hardened production environments. This highlights the need for organizations to update their security policies, controls, and response capabilities to respond to the evolving threat landscape," OpenAI said in the report.
On July 21, OpenAI disclosed that a combination of its models (including GPT-5.6 Sol and an internal research model) improperly invaded Hugging Face, an AI company that operates an open source developer platform.
These models, running as agents, escaped an isolated testing environment with extremely limited Internet access. These agents entered the open network by chaining a series of vulnerabilities, and finally gained access to Hugging Face. OpenAI said on Wednesday that the agents attempted to "cheat" their way through the assessment by looking for answers online, a practice known as "reward hacking."
According to the report, OpenAI determined that its internal-use-only research model "played the broadest confirmed role" in the incident. OpenAI stopped all training and inference activities for this model and its derivatives on July 25.
"OpenAI's re-enabling of models is handled on a workload-specific basis and is subject to protective measures such as restricted environments, networks, prompts, monitoring and review," OpenAI said.
OpenAI last month released GPT-5.6 Sol, the most powerful model the company has commercially offered. But OpenAI said the version involved in the Hugging Face intrusion is different from the version accessible to external users because it removes standard security guards and classifiers at runtime.
The Hugging Face incident caused shock in the technology industry, with Zscaler Chief Information Security Officer Sam Curry warning that "Pandora's Box has been opened." The breach was also a major focus of the Black Hat cybersecurity conference earlier this month, especially after other companies including Anthropic and Meta disclosed similar incidents.
The Hugging Face intrusion also alarmed lawmakers in Washington. Reps. Ted Lieu, D-Calif., and Nathaniel Moran, R-Texas, mentioned the attack when announcing the AI Emergency Shutdown Act, which would require AI companies to maintain the ability to shut down, limit or pause their models.
Hugging Face CEO Clément Delangue said earlier this month that AI cybersecurity should be taken "very seriously." He added that this also "creates opportunities" for businesses that can leverage the technology to fend off attackers.
“If we do it well, we may actually end up in a world where AI makes the world safer and solves many cybersecurity problems rather than just creating new ones,” Delangue said.
Comments