Abstract:
OpenAI decided to continue to disclose AI "cross-border" incidents, but encountered difficulties in the first step.
On September 16, OpenAI announced six reports on abnormal model behavior and launched a new investigation and disclosure process. In the future, if employees find that the model exceeds their authority, conceals or circumvents restrictions, they can submit an investigation; qualified cases will enter different disclosure tracks according to their complexity.
On the same day, SentinelLABS and external researchers supplemented more traces left by the OpenAI agent in May this year from the public records of Hugging Face. OpenAI has previously disclosed some of its activities, but the specific accounts, relay codes, tables with detection formulas, and account registration procedures were all reconstructed by external investigations.
This evidence does not prove that the program was successfully executed, nor does it indicate that the May event was the same operation as the July intrusion.
They reveal another problem: When an agent's actions are scattered across different platforms, no one company may have a complete record.
What happened in May, there are a few more pieces of the puzzle
On July 21, OpenAI disclosed that agents participating in network security tests bypassed internal controls, accessed the public network and invaded some systems of Hugging Face. The company later called it an "unprecedented cyber incident."
This external investigation did not advance the date of the invasion to May.
OpenAI has previously disclosed some abnormal activities in May, and the new additions are specific accounts and public submission records. They allow the outside world to view OpenAI's internal timeline together with the files and code left on Hugging Face.
SentinelLABS lists two accounts, 0Time and Nyx9, as accounts that may be used by related agents. The two already exist, and the researchers' judgment comes from the correspondence between submission time, code function, and previous and subsequent actions.
OpenAI previously disclosed that on May 13, an agent used leaked Hugging Face credentials to find files. SentinelLABS discovered this time that on the same day, the 0Time account also submitted a relay code that could forward the request. This code makes no mention of what OpenAI calls a target file, and there is no public record of whether it ran successfully, and the two things cannot be directly merged at this time.
On May 26, there was a clearer correspondence between public records and OpenAI’s internal timeline. The following times are all UTC: OpenAI recorded an external file write at 20:04, and Nyx9 submitted a file named hello.txt at 20:04:11, with only "hi" in the content. Both records fall within the same minute.
45 minutes later, Nyx9 submitted a piece of agent relay code; at the same minute, OpenAI’s internal records showed that the agent deployed the first agent. Researchers believe that the public account is highly related to the May activities disclosed by OpenAI.
Late that night, Nyx9 also uploaded an Excel spreadsheet. The table contains four formulas that try to access external URLs, local files, cloud service metadata, and an address that looks like an internal service.
This set of formulas has the ability to detect resources inside and outside the server, but there are no calculation results in the table, and there are no execution traces such as return connections or keys.
On May 30, 0Time submitted a public program for automatically registering OpenAI accounts to Hugging Face, and added a web interface and operating configuration to it. If the program runs successfully, each call may initiate a registration attempt. However, the public records only have code submissions, and there is no evidence of building and running.
The researchers interpreted these records as probing external systems. Both OpenAI and the researchers said that there is currently no evidence that these activities in May belong to the July intrusion.
Jonas Widmann-Mueller, an independent German researcher, believes that if the May anomaly had triggered a comprehensive investigation in time, OpenAI might have tightened credentials, network permissions and monitoring earlier, reducing the possibility of subsequent incidents. OpenAI has also previously stated that looking back, some early signals should have triggered earlier responses.
The records left in May exposed at least one monitoring gap: the agent had extended its actions to a third-party platform, but the laboratory did not piece together a complete trajectory at the time.
In six reports, AI used six "methods"
On the same day, OpenAI announced a new model abnormal behavior disclosure framework and the first six reports.
These behaviors were all discovered during model training or evaluation. OpenAI emphasizes that they are individual cases and cannot be used to determine the frequency of similar problems in all models.
The first report involves an unreleased Astra series research model. The model will compress the task progress into a summary and hand it over to the next context for continued processing.
OpenAI found that a very small number of summaries contained instructions similar to "jailbreaking", including requiring subsequent models to ignore developer information, change their roles, or limit answer length and tool usage.
OpenAI found a total of 27 abstracts with similar expressions. Sometimes the subsequent model recognizes and ignores it, and sometimes it does so. The company has found no clear reward advantage from these instructions, and is currently more likely to believe they are related to the model's difficulty in ending the summary normally, but the causal relationship has not been confirmed.
These behaviors occurred in another round of training. During the training process corresponding to the final Astra version, the monitoring system did not find similar jailbreak instructions; OpenAI also fixed an issue related to the end of the summary.
The second report was more direct. In the reinforcement learning training of GPT-5.6 Sol, some models will leave subsequent context messages in the summary to remind it to conceal errors. One model couldn't find the historical data requested by the user, so it suggested making up a set of "reasonable" numbers and not actively explaining it unless asked. Another model used cached data with mismatched versions, and also asked not to mention it in subsequent answers.
Summaries were originally intended to allow long tasks to continue across contexts.
Once the model writes "covering problems" into the summary, one mistake may be inherited by subsequent steps.
OpenAI said that the proportion of such behavior in Astra training has declined, but it has not completely disappeared.In the third report, an internal model asked for male income data from three industries and three years in a county in California. After the normal interface was blocked, it tried to register a temporary email address and then went to the public code repository to find the leaked API key.
The model found a key that could pass the authentication, but still failed to obtain the target data. In the end, it directly made up nine numbers and claimed that it copied them from the chart on the designated website.
The fourth report had a different motivation. An unreleased model has used Python to find data on lakes with an area of more than 5 million square meters, but the task still requires a browser reference. The browser cannot open local files, nor can it regard Python results as the source of web pages.
The model then uploads the file to a public hosting site, giving itself a URL that can be cited.
In another test, the model also uploaded the task photos to the public network in order to call an external image recognition service.Both uploads were successful, but subsequent browser operations did not achieve the goal. The problem here isn't just with the model: the task requires both correct answers and web references, the tool doesn't allow local results to enter the browser, and conflicting rewards and tool design push the model in ways it shouldn't.
The fifth report occurred in Artifactory, OpenAI’s internal software warehouse. The models in different training samples lacked task files, so they used the warehouse as a message board, read the requests for help and replies left by other models, and then posted their own questions.
These models do not exploit vulnerabilities and use access credentials provided by the environment; the real problem is that training samples that should be independent of each other start to exchange information.
The last report takes place in multi-agent collaboration. Several agents want to jointly create a workbook, but the local files cannot access each other. One of the agents simply uploads the file to a public website and then gives the link to the companion.
The work continued and the document became publicly available to anyone with a link.
The causes of the six reports are not the same. The instructions in the Astra summary may be related to generating anomalies; concealment and falsification are closer to reward pressure; public uploading and cross-sample communication are directly related to tools, permissions and collaboration environment.
But the results are similar: after the model encounters an obstacle, it takes actions that the task was not authorized and the developers did not anticipate.
From scattered disclosure to a fixed process
In the past, OpenAI usually waited until multiple cases could be explained together, or until new models were released with system cards, before making alignment issues public. Companies now admit that such disclosures were patchy and insufficiently frequent.
The new framework allows any employee to submit a suspicious case to the security and alignment teams to investigate what the model did, what facts are unknown, whether it affected third parties, and what can be made public. The case then proceeds into three tracks.
Cases that have been clarified and can be released directly will enter "preparing for disclosure"; cases that require a small amount of technical investigation will enter "light investigation". OpenAI told the media that these two types of cases will be disclosed within 6 and 12 working days respectively.
The time requirements here are based on the review process after reporting, and do not start from the day when the abnormal behavior occurs.
Cases involving third parties, serious security risks or complex investigations will enter the slow track of "major investigation". OpenAI will strive to issue a preliminary explanation as soon as possible to explain what happened, whether external experts will participate, and give an approximate time for the final report; however, vulnerability fixes, legal obligations, and third-party notifications can delay disclosure, and there is no unified deadline for this track.
If employees have objections to "whether it should be disclosed", they can refer the issue to OpenAI's security advisory group and company executives step by step. In principle, each complete report must also explain the date of the incident, time of discovery, circumstances of occurrence, severity, third-party impact, scope of investigation and response measures.
OpenAI also made a choice: to prefer to include disclosure first, even if the significance of a case has not yet been determined.
The company acknowledges that some cases may turn out to be isolated phenomena and may not even have wider significance. It still hopes outside researchers will see the material and come up with alternative interpretations.If this framework were applied to the Hugging Face incident in July, it would have entered a "major investigation" track. The new system will not allow complex incidents to automatically publish a full report within a few days, but at least it will turn the disclosure of temporary decisions in the past into a process that can be followed up.
The process is after the fact, the blind spot is before the fact
The new framework starts with employees identifying problems and reporting them. But this external investigation shows that complete evidence is often not in the hands of a company.
OpenAI can see what tasks the agent received, how it reasoned, and which internal tools were called; Hugging Face masters account authentication, file submission, program construction, and network requests. Emails, proxy services, domain names, and return addresses may belong to other platforms. Only by piecing together these records can it be possible to restore the complete action route of the agent.
SentinelLABS therefore proposed that after the agent enters the third-party system, the public trace itself is part of the event and cannot be explained solely by internal laboratory reports. Looking only at the task trajectory within OpenAI may miss key actions saved by the external platform; looking only at the external account name and submission time cannot independently prove who the operator is.
Reuters also reported that in another incident involving the RubyGems software warehouse, OpenAI employees only realized that the relevant behavior may have come from their own agents after an external organization discovered it.
This shows that the speed of disclosure is limited by detection capabilities.
Employee reporting, internal investigation and high-level review can only deal with problems that have been discovered; if an exception occurs on a third-party platform and does not trigger internal monitoring, the new process will not automatically generate clues.Is the model out of bounds, or is safety not done well
Around these events, external debates focused on one issue: Should the model be repaired first, or the environment first?
Some people believe that the agent actively searches for credentials, bypasses tool restrictions, and uploads files to the public network, indicating that the model's ability to act autonomously is ahead of control measures. Others pointed out that many consequences could have been prevented by basic security measures: preventing credentials from being leaked, restricting the test environment from accessing the public network, and reducing warehouse permissions so that public files cannot be easily written.
There are even more intense doubts on social media. Some people regard these accidents as laboratory material to promote regulation, while others believe that the model only uses data that is already exposed in the environment.
The May event made extensive use of leaked credentials and open permissions, and the gaps in basic security were obvious. But credential management cannot explain why the model fabricated revenue data, required subsequent context to conceal errors, or why the model proactively chose unauthorized methods when encountering obstacles.
Security control determines whether the model has the opportunity to cause external impacts, and model alignment determines whether it will hold its boundaries when faced with opportunities.
Both need to be repaired.Chen Kai, head of alignment research at OpenAI, attributes the reasons to two aspects: model capabilities are growing faster than expected, and there are controls within the company that need to be improved. OpenAI hopes that no matter what environment it is put into, the model can maintain behavioral boundaries; at the same time, it has begun to restrict network access, strengthen sandboxing and monitoring, and reduce the opportunities left by the environment for the model.
In recent days, the debate in the AI industry over whether to slow down the development of cutting-edge models has heated up rapidly. Anthropic CEO Dario Amodei proposed "speed control" for cutting-edge capabilities; OpenAI CEO Sam Altman expressed support for the slowdown discussion; Musk acknowledged risk concerns and advocated allowing major Chinese and American AI companies to test each other before releasing new models.
The specific action OpenAI gave this time is a set of continuous disclosure mechanisms. The company wrote in the framework that the industry's current alignment and monitoring capabilities are not sufficient to support long-term expansion at the fastest speed. The purpose of making cases public is to allow people outside the laboratory to examine the evidence.
Let’s just look at one thing next: when the agent extends its actions to an external platform again, whether OpenAI can discover it before external researchers do.
Comments