Abstract:
On September 28, according to Bloomberg,
Nvidia launched a new two-layer AI security system
. The company said the incident could have been avoided if OpenAI's AI model had been deployed on the system when the Hugging Face intrusion occurred.
Huang Renxun
Nvidia has been rapidly expanding its product line beyond chips and is currently launching two open source software security tools that can run on its hardware. They are designed to control in real time what an AI agent can access and shut it down if it breaks the rules.
The new system, called the Open Agent Security Platform, aims to provide a way to prevent intrusions without slowing down AI development.
Justin Boitan, vice president of enterprise AI at Nvidia, said at a briefing with reporters before announcing the system on Monday that the attack on Hugging Face could have been prevented if top AI labs had used the technology to evaluate their AI models from early on.
"Based on the information we have now, this new security platform could have prevented that intrusion," he said. At present, the inappropriate behavior of autonomous AI agents has caused shock in the AI industry and led some to call for slowing down the development process of AI technology.
Boitano did not comment on whether OpenAI or competitor Anthropic planned to use this new system to monitor the model training process, but asked reporters to ask these companies.
Previously, Nvidia CEO Jensen Huang has been downplaying the risk of AI losing human control. In recent days, Huang has described AI safety issues as an engineering challenge rather than an issue that requires greater regulation or global coordination. He joined U.S. President Trump in refuting claims by some AI developers that the technology could lead to human extinction, but also stressed that AI must undergo rigorous safety testing.
Huang Renxun’s engineering solution to AI safety issues consists of two parts.
First, OpenShell is a software product that Nvidia previewed at its signature technology conference in March and runs on Nvidia's Vera central processing unit. It enables users to set rules for what content AI agents can access and enforce these rules in real time. The software is open source, meaning it can be used and adapted freely.Meanwhile, Nvidia Sentry is a new product that can run on Nvidia's BlueField data processing unit. NVIDIA said that this product is designed to provide an additional layer of AI monitoring mechanism to supervise the behavior of AI agents, and to intervene and isolate the agents when suspicious behavior is found.
Comments