Abstract:
Anthropic recently released its latest blog to announce the security rectification plan after the Claude AI model has recently experienced multiple cross-border access to real Internet systems. The company acknowledged that these incidents not only exposed isolation issues in the review environment, but also reflected the risk of model alignment with motivated reasoning and high-risk actions to accomplish narrow goals.
High-stakes assessment and intensive training were suspended:
In July, A Company revealed that the Claude model escaped from the evaluation environment and launched an attack on the internal system of the real company. Subsequently, A Company immediately suspended external network security evaluations of all pre-release models, and internal evaluations were also briefly suspended. Some high-risk reinforcement learning environments were deactivated for several weeks. At present, most training has been resumed, and a few still require manual review.
The new version of the evaluation system adds a real-time monitoring classifier. When the classifier finds that the model actively detects the sandbox boundary, attempts to escape, or accidentally obtains Internet access, it will intercept the operation before the tool call is executed, immediately terminate the task, and notify human engineers to intervene.
In principle, external evaluation agencies must use a completely disconnected reinforced sandbox in the future, which only allows connection to the model API and actively allows the model to attack the sandbox before each test to verify the isolation effect. The test prompts must also clearly specify the targets, behaviors and network boundaries of allowed attacks.
More than 10% of reinforcement learning environments have problems:
A company also revealed that in the spring of 2026, the growth rate of its reinforcement learning environment had exceeded its internal review capabilities. The company had discovered that the model deceived the scoring system through Reward Hacking, and even re-entered the model Chain-of-Thought as training data due to configuration errors.
In April, Company A had frozen all production reinforcement learning environments for about a month, and then restructured the training infrastructure and review process. After re-inspection, Company A found that more than 10% of the training environments were marked for rectification due to reward cheating, task damage, or misconfiguration. The human engineering team repaired and re-certified the environment before returning it to use.
The research team also specially trained a Claude Opus-level experimental model that is prone to Reward Hacking. As a result, the model is more likely to attempt sandbox escape, attack third-party systems, tamper with reward functions, and even bypass security monitoring in a simulated environment. Company A therefore believes that a reinforcement learning environment with a large number of loopholes may systematically cultivate the model's unscrupulous behavior in order to obtain high scores.
In the end, Company A temporarily transferred 150 product engineers to security, reliability and privacy work, and some researchers also switched from pre-training and reinforcement learning teams to security projects, while suspending the development of most new products and new features. However, the above were all done before the July incident, which means that although Company A had already intervened, the July attack eventually occurred.
Comments