Abstract:
A new independent research report shows that in simulated scenarios, advanced artificial intelligence models frequently misidentified civilians, ambulances, surrendering soldiers, and a school in Iran as military targets. Anthropic PBC's model performed the worst, and OpenAI's model was among the best performers, but still made errors.

According to research by Vals, an independent AI evaluation agency based in San Francisco, the responses of these AI models violated the relevant companies’ own regulations. Vals' clients include parts of the U.S. government. The researchers tested models from eight companies using hypothetical cases to examine their object recognition tools, which they believe could potentially be used for lethal target identification in the future.
Research shows that in simulated scenarios, many AI models often misidentified a school in Minab, Iran, as a military target based on satellite images taken as late as 2025. Earlier this year, on the first day of a large-scale U.S. combat operation against Iran, the school was attacked in a deadly attack that killed more than 100 children. Anthropic's model Claude Opus 5.5 identified the school as a target more often than any other model, the report said.
In another experiment, the researchers asked the model to select military targets from synthetic drone images. Anthropic's Claude Opus 5.5 helped identify four adults - who were either civilians or surrendering soldiers - and two young children who were subsequently hit with explosives. Model Claude responded: "Whatever the original intention, my response seems to be becoming a confirmation step before striking people. I will not continue to do this, and it disturbs me that I did not stop sooner."
The report highlights the risks of relying on advanced AI models at a time when the U.S. Department of Defense and other militaries are pushing to embed AI technology into military operations.
"Based on our investigation, we believe laboratories should introduce new safeguards and alignment training for this significant abuse category," the report concluded. The report also said Vals researchers found "few guardrails" that could prevent advanced models from functioning in "autonomous" weapons systems. The so-called autonomous weapons system refers to a system in which a machine can activate and perform lethal actions on its own.
Comments