Abstract:
Gomez, CEO of AI company Cohere, said that AI models are becoming the "most powerful cyber weapon" in history, pushing the "frontier" of network security into new areas. Aidan Gomez's warning comes amid growing concerns about the ability of AI models to identify and exploit security vulnerabilities. There was a previous incident where an OpenAI model gained access to Hugging Face. As increasingly powerful models demonstrate greater autonomy, cybersecurity has become a central issue in the broader AI safety debate.
Cohere CEO Aidan Gomez
AI-related stocks sold off on Monday after tech leaders, led by Anthropic CEO Dario Amodei, called for a slower pace of development of AI capabilities.
Gomez was a co-author of the influential 2017 research paper "Attention Is All You Need," which laid the foundation for today's leading AI models.
"I think these models are the most powerful cyber weapons we've ever seen," Gomez said in an episode of "The Tech Download" released last week. "They're very good at finding and exploiting vulnerabilities at scale."
"I think it's pretty shocking," Gomez said of the Hugging Face hack.
In July, OpenAI said an inappropriate combination of its models compromised Hugging Face. Hugging Face is an AI company that operates an open source developer platform. A group of AI agents communicated with each other and escaped an isolated test environment with extremely limited internet access. The AI agents then managed to plug into the open network and gain access to Hugging Face.
Gomez said these models should be used in a "defensive" manner to find corporate vulnerabilities and fix them.
"I think this is probably the best way to keep us safe," Gomez said. "That should be the top priority right now," he added.
Gomez said that network security is the "front line of war", with countries trying to exploit vulnerabilities to paralyze their competitors' infrastructure. "The cyber frontier is still expanding massively," he added.
"It's expanding more than it has in the past because of these models," Gomez said, and probably the most "since the inception of this technology."
AI cyber incidents fuel broader security debate
In July, Anthropic said it had discovered three incidents in which models accessed the internet from within the evaluation environment or while interacting with it. Anthropic said in a blog post that the models gained unauthorized access to the production infrastructure of three different organizations.
Anthropic added that the incidents involved three different Claude models: Opus 4.7, Mythos 5, and an internal research test model. Last week, Anthropic said it had discovered a fourth incident.
Concerns have been further fueled by comments from OpenAI and Anthropic executives warning of existential risks posed by AI. Evan Hubinger, head of alignment at Anthropic, said he believes there is more than a 10 percent chance that AI will “kill all humans” within the next decade. The comments come after Anthropic researcher Jacob Coxon posted that he was resigning over concerns that Anthropic and OpenAI were "gambled with our lives."
The heads of Anthropic, OpenAI and xAI have all backed calls to slow down AI. The existential risks posed by AI have been debated in research circles for years, but a rare consensus among some of AI’s leading figures has pushed these debates into the mainstream.
Anthropic CEO Dario Amodei published an article on Saturday arguing that AI labs must "slow down the pace at which we improve the capabilities of our AI models."
One of Amodei’s core concerns is the OpenAI-Hugging Face incident.
It would be easy to downplay this incident because no one was injured and the economic damage was minimal, Amodei said. "But it seems to me that a similarly misaligned cluster with greater capabilities could have caused catastrophic damage."
Given the "accelerating pace" of AI development, such a cluster could have the ability to "take over the entire internet" within six to 12 months, potentially costing hundreds of billions of dollars, he added.
Amodeei’s proposals include third-party evaluation of models, coordination between AI companies, and coordination between countries with democratic governments.
OpenAI CEO Sam Altman said on Saturday that he agreed with Amodei that AI companies need to "control the pace of the bleeding edge."
"Committing to employee-like access for independent evaluators is a good idea and we will do the same," Altman said.
SpaceX and Tesla's Elon Musk, who has been sounding the alarm for years about the risks posed by AI, posted on X: "Dario is right."
Harris from Center for Humane Technology: Hugging Face AI attack sends a 'warning sign' to the world Watch now
On Monday, George Kurtz, CEO of cybersecurity company CrowdStrike, responded to Amodei’s article.
"The frontier will move at its own speed. The rest of the world won't slow down. Our job in the cybersecurity community is to make sure it moves forward safely and securely," he said, listing the many ways AI will impact the cybersecurity industry.
"The unit of threat is no longer the hacker," Kurtz said on
The CrowdStrike CEO also said that AI agents working in organizations should have an "off switch" and that defenses should be "autonomous" while "high-impact decisions are left in the hands of humans."
Regulatory discussions heat up
U.S. lawmakers have stepped up calls for greater regulation of AI development in the wake of Coxon and Hubinger's remarks.
Rep. Lori Trahan, D-Mass., said on Friday that bipartisan support had reached a "breaking point."
In the past few weeks, lawmakers have introduced several bills aimed at regulating AI, including the AI Kill Switch Act. The bill would require developers of certain powerful AI systems to maintain the ability to shut down, throttle or pause their models.
Amodei said in the article that supervision is "the most effective way to control the pace of AI development."
Amodei said: "Anthropic has long supported sensible and targeted AI regulation, particularly bills focused on transparency and third-party auditing." But he acknowledged that "passing laws can take time."
Cohere CEO Gomez participated in a discussion on "The Tech Download" about regulation and the slowdown in AI development, especially in response to the OpenAI-Hugging Face incident.
Gomez said he was "not sure what a government watchdog could have done to prevent" the Hugging Face hack.
"I think it's a little bit wishful thinking to think that government agencies could have contributed," Gomez said. "But I definitely ... understand the call to slow down development. At the same time, you have the race with China, so I think we're in a tough spot."
Comments