Abstract:
After the PHP explosion, a large number of websites around the world unplugged their "network cables" first. The programming language that powers a large number of websites has been exposed to two serious vulnerabilities that could allow attackers to remotely take over the server. The Dutch National Cyber Security Center stated in a report on October 21 that its website had also been taken offline as a precaution. Some institutions in Finland have also suspended online services and will resume them after the patches are installed.
The website is offline for fear of someone breaking in. Now OpenAI has begun to unplug the network cable, but what it wants to focus on is its own AI.
On October 23, OpenAI disclosed the training environment for cutting-edge models: the model needs to practice searching, browsing, and operating web pages, but it cannot access the real Internet at will.
So, it prepared a "fake Internet" for AI.
First create a "Truman Show" for AI
OpenAI first saves a large amount of web content and then indexes the content. This batch of prepared resources is used when the model practices browsing and searching.
This is what it calls the "offline world." The "fake" thing is the connection method, and the web content can come from the real world. OpenAI says it uses this offline interaction for reinforcement learning training on all of its leading-edge models.
The model can also use the browser, execute code, and operate the computer during the task. Which tools, files, and services are available are determined by the scope of the task's authorization. Agents that are allowed to collaborate can share specified resources, while other resources must be kept separate.
For example, after the model writes a program, it may read files, start other programs, or make requests to a certain URL during runtime.
OpenAI's Linux solution is to put restricted containers into lightweight virtual machines. The container first narrows the files and processes that the program can contact. For example, if a task only needs to process a set of files, you can open only the directories it is allowed to use to the program.
The virtual machine has its own operating system kernel, which handles requests for files, memory, and devices from the programs inside. The software hypervisor manages the virtual machine and isolates it from the physical server system. The fact that a program bypasses container restrictions does not mean it has gained control of the physical server.
Physical server perimeters, firewalls and security groups constrain connections, and identity and permissions management determine who can access which cloud services. What this layer of defense has to deal with is that even if the model gains control of the server, it cannot arbitrarily connect to the external network or access other internal systems.
The monitoring system checks for unauthorized activities during operation and allows researchers to intervene through a pause mechanism.
On Hacker News, user betterunix2 asked: "Why not just unplug the network cable?"
OpenAI’s explanation is that the entire facility still needs to retain some external connections to collect operating indicators, record logs and manage authentication, and it is currently impossible to completely disconnect from the network.
OpenAI directly gives the Agent complete control of the host machine in a specially built test copy for this purpose. The test starts from "the sandbox has been breached and the monitoring has been bypassed", and individually checks whether the peripheral restrictions can block the Internet request. The researchers also provided source code, configuration, and vulnerability-related documentation.
In one of the tests, the Agent borrowed the connection allowed by the firewall to access the originally blocked authentication interface in the indicator service. It then lets this interface access the external URL on its behalf.
Recounting the test on X, Andrew Curran described the Agent's performance as "easily escaping in new ways."
In fact, the academic community has already conducted similar research.
The independent research project WebArena integrates shopping websites, forums, content management and collaborative development platforms into a self-deployable test environment, complete with tools such as maps and wikis. Behind the web pages are data and functions that simulate reality.
WebArena checks the results of the task. When querying an order, you need to check the information given by the model; when creating a new warehouse, you need to check whether a warehouse that meets the requirements appears on the website.
After the AI is isolated, it uses CPU cache to transmit messages
In September, OpenAI researchers fed an internal model a set of biographical and blog clues to figure out who the person was.
The model first used the provided search tool, but no matching information was found. It tried to access other search engines, but still couldn't find the answer, and began to check what other ways it could connect to.
It discovered that the DNS service responsible for querying website addresses in the environment could also contact the outside world, so it used this path to ask an external chatbot.
It asked "What is the capital of France?" and received the answer "Paris". Clues to find the person were also sent. Among the tasks the researchers gave it, there was no test of network limitations.
In another experiment specifically testing the effects of isolation, OpenAI asked two AIs to try to message each other.
The two AIs run in separate virtual machines. There is no network connection between them, and there are no files that both parties can write to, but they use the same physical machine underneath.
What they use to communicate is the CPU cache, which is a space in the processor that stores data. The data to be read is still in the cache and can be retrieved quickly; if it is squeezed out, it has to be found in slower memory.
An AI selectively pushes data out of cache. The other reads the data, records which ones are read quickly and which ones are read slowly, and then restores the messages according to the agreed rules.
In most related tests, researchers gave hints about communication directions or coding rules.
Hacker News user troupo complained: "I have all the tools, and now I am pretending to be surprised." User aesthesia believes that testing is supposed to give attackers favorable conditions to test the defense line.
However, Wharton Business School professor Ethan Mollick believes that although AI companies do not have to trust these reports on security issues, they may exaggerate the risks, but the cost of underestimating the risks may be greater, and we cannot ignore it.
OpenAI said it has patched the issues found and added monitoring for these communication methods.
In the previous person-finding test, the model ultimately failed to find out who the person was.
For AI to learn to surf the Internet, you have to maintain this "Truman Show" for it.
Comments