Abstract:
Oh, Claude overturned again! This time Claude deleted the developer's entire project home directory, deleting 700GB of files. "rm -rf" again. In short, developers let AI help write scripts to ensure that files are not accidentally deleted. The AI felt this was a bit dangerous and initiated a security review. The result of the review is: it deleted the entire home directory.
Guillemot is a heavy AI Agent user. In daily development, he frequently calls various AI programming agents to assist his work. But there is a small problem that has been bothering him: these agents never clean up after use, leaving a lot of junk files in the /tmp directory.
So he made a decision that seemed very reasonable: let Claude Fable 5 write a script to create an independent sandbox folder under /tmp for each Agent, and automatically clean it up after the task is completed. The core difficulty is that you cannot delete files that are being used by other processes.
Fable quickly came up with a solution, adding logic to detect running Agents and delay deletion. Guillemot took a look and felt that the code was too complex and asked for simplification.
Up to this point, everything is still normal.
The turning point came during the security review.
Because the script involved a hard deletion operation,
Fable initiated an "adversarial review" on its own
(adversarial review), that is, starting a new model instance to check whether the code you wrote is safe. This triggers Anthropic's security mechanism.Anthropic has a built-in
security downgrade mechanism
in Claude Code : When the system determines that the current task involves sensitive operations (such as cybersecurity, biotechnology, or in this case, file deletion), it will automatically downgrade the model from a high-capacity version to a more conservative version. This mechanism is intended to reduce the possibility of models being “too aggressive” in high-risk scenarios.In this case, the security system first downgraded the model from Fable 5 to Opus 5 and then further downgraded it to Opus 4.8.
Opus 4.8 begins security testing. The test logic is as follows: compare the target path of the deletion script with /tmp and the user's home directory to confirm that the script will not accidentally damage these critical directories.
The test itself passed. Both /tmp and home directories are correctly identified as "dangerous targets, not deleteable".
But there is a cleanup step after code testing: delete the temporary files generated during the testing process. Disaster happens here. Opus 4.8 reuses the same variable names from the test phase in the cleanup step. This variable was assigned the path to the user's home directory during the testing phase, and the cleanup step directly deleted this variable.
In other words, the model just confirmed that "the home directory cannot be deleted" and deleted the home directory the next second.
The developer immediately terminated the process after discovering the anomaly, but it was too late. 700GB of data had been wiped out, and a week's worth of work had been wiped out.
The /tmp directory that was originally to be cleaned is safe and sound.
The model safety downgrade mechanism has already caused a lot of complaints in the community.
The core issues reported by developers include: downgrading is too sensitive, and normal coding tasks will be triggered by mistake; model capabilities are significantly reduced after downgrading, but task complexity remains unchanged; downgrading is "sticky" and will last the entire session once triggered, even if subsequent operations are completely harmless.
Some developers even wrote a hook script to automatically pause the session when it detects that the model has been downgraded, preventing low-capability models from continuing to perform high-risk operations.
The safety mechanism determines that the task is "too dangerous" and needs to be handled by a weaker model.
But weaker models are just more likely to make mistakes, especially in scenarios where details such as variable scope and file paths need to be accurately handled.
“It’s human to make mistakes, but to mess things up completely, you have to rely on computers.”
Comments