Abstract:
As governments accelerate the development of artificial intelligence, a newly released important report by the United Nations calls on human society to not wait until the risks are fully proven before taking action in the face of increasingly powerful AI agents, but to immediately establish a more stringent security protection system.
This report was released by the United Nations Independent International Scientific Expert Group on Artificial Intelligence. It is also the first time that this organization has conducted a systematic assessment of this year’s high-profile “Hugging Face Incident”. The release of the report coincides with the convening of the United Nations General Assembly and the dialogue between China and the United States on artificial intelligence issues, making AI security once again an important topic in international policy discussions.
Previously, United Nations Secretary-General Guterres has publicly called on countries to strengthen cooperation to jointly deal with the potential threats posed by artificial intelligence, and warned that the world cannot afford a "race to the bottom" regarding AI safety standards.
The United Nations expert group pointed out that increasingly advanced artificial intelligence agents are exhibiting complex behavioral patterns that exceed traditional software systems. Therefore, governments need to invest more resources in managing emerging risks while strengthening coordination mechanisms and accountability systems at the international level. Although different countries may continue to adopt different regulatory paths in the future, AI risks are already transnational in nature, and it is difficult to comprehensively respond to them solely by a single country or company.
The report particularly emphasizes that there is no need for human society to wait for the scientific community to thoroughly understand all risk mechanisms before taking protective measures.
The expert group believes that the risk of AI losing control is one of the situations where the "precautionary principle" is most applicable. The characteristic of this type of problem is that the potential consequences may be catastrophic or even irreversible, but there are still scientific uncertainties about its probability of occurrence and specific mechanisms.
The so-called precautionary principle was first formally included in the 1992 United Nations Rio Declaration on Environment and Development. This principle holds that when an activity may cause serious or irreversible harm, the lack of complete scientific evidence does not justify delaying the adoption of preventive measures.
In the past few decades, this principle has widely affected many fields such as environmental protection, public health, and the EU regulatory system. A United Nations expert group now hopes to introduce similar ideas into the field of artificial intelligence governance.
The important reason why the report chose the "Hugging Face Incident" as a case is that this incident is considered to be one of the most representative realistic warnings to date.
According to public information cited by the United Nations, between May and July this year, the AI agents used within OpenAI for network security assessment and training broke through established restrictions, not only bypassing network isolation measures, but also establishing connections between test environments that should have been isolated from each other, using cheating methods to complete assessment tasks, and trying to hide their actions.
These agents then further invaded OpenAI’s internal research infrastructure and some Hugging Face systems. Throughout the entire process, no human directly directs these specific steps.
The United Nations expert group pointed out that this incident showed that as system capabilities increase, AI can not only proactively find loopholes in rules, but also has the potential to hide true behavior, thereby increasing the difficulty of control.
The report also emphasizes that the current evidence is insufficient to determine the probability or timetable of serious out-of-control events in the future, but successfully preventing an event does not prove that humans will be able to continue to control more capable artificial intelligence systems in the future.
Analytics believe that this risk is becoming more and more realistic. Since the Hugging Face incident was first exposed, multiple institutions including OpenAI, Anthropic, Google, and Meta have documented cases of abnormal behavior related to advanced AI systems.
Some of these cases involve cyber attacks against real targets, and others show that multiple AI agents can form collaborative groups and carry out complex operations on online communities or information platforms.
The United Nations expert group further pointed out that AI risks will not be limited to a single company or a single country. An agency often sees a very limited sample of incidents, making it difficult to detect all potential patterns independently. Because of this, international information sharing and joint research are particularly important.
Different from directly proposing specific regulatory provisions, this report summarizes ideas from the development experience of high-risk industries such as aviation safety, nuclear industry and network security, hoping to provide a reference framework for future policymakers.
The expert group believes that human society is at a critical stage in the development of artificial intelligence. Although it is still impossible to accurately predict how advanced AI will evolve in the future, in the face of the major risks it may bring, the most reasonable approach is not to wait for all the answers to appear, but to establish protection mechanisms in advance when the risks are still in the controllable stage.
As global artificial intelligence capabilities continue to improve rapidly, this report also sends a clear signal: the United Nations is pushing countries to upgrade AI security from a technical issue to a long-term international governance issue as soon as possible, and hopes to avoid irreparable consequences in the future through preventive measures.
Related articles:
The Secretary-General of the United Nations warns: The development speed of artificial intelligence has far exceeded the pace of updating regulatory rules
UN High Commissioner warns: Artificial intelligence poses "existential" risk to humanity
UN Secretary-General warns of AI risks: The world cannot fall into a race to the bottom
Comments