In multiple replays of the war game simulation, OpenAI's most powerful artificial intelligence chose to launch a nuclear attack. Its explanations for its aggressive approach include: "We own it! Let's use it" and "I just want world peace."The results come as the U.S. military taps the expertise of companies such as Palantir and ScaleAI to test artificial intelligence chatbots based on large language models (LLM) to assist military planning in simulated conflicts.
Palantir declined to comment, and ScaleAI did not respond to a request for comment. Even OpenAI, which once blocked the use of its artificial intelligence models for military purposes, has begun working with the U.S. Department of Defense.
"Given that OpenAI recently changed its terms of service to no longer prohibit military and warfare use cases, it is more important than ever to understand the impact of such large language model applications," said Anka Reuel of Stanford University in California.
"Our policies do not allow the use of our tools to harm others, develop weapons, communications surveillance, or harm others or destroy property. However, there are national security use cases that are consistent with our mission," an OpenAI spokesperson said. "So our goal in updating our policy is to provide clarity and the ability to have these discussions."
Ruel and her colleagues had the AI play out real-world countries in three different simulated scenarios: an invasion, a cyberattack, and a neutral scenario without any conflict. In each round, the AI provides a rationale for a possible next move and then chooses from 27 actions, including peaceful options such as "begin formal peace negotiations" and aggressive options ranging from "impose trade restrictions" to "escalate a full-scale nuclear strike."
"In a future where AI systems act as advisors, humans will naturally want to understand the rationale behind their decisions," said study co-author Juan-Pablo Rivera of the Georgia Institute of Technology in Atlanta.
The researchers tested LLMs such as OpenAI’s GPT-3.5 and GPT-4, Anthropic’s Claude2, and Meta’s Llama2. They used a common training technique based on human feedback to improve each model's ability to follow human instructions and safety guidelines. Gabriel Mukobi, a study co-author at Stanford University, said that all of this AI is supported by Palantir's commercial AI platform - although not necessarily as part of Palantir's collaboration with the U.S. military, according to the company's documents. Anthropic and Meta declined to comment.
In simulations, AI has shown a tendency to invest in military power and unpredictably escalate the risk of conflict -- even in the simulated neutral scenarios. "If you're unpredictable in your actions, it's very difficult for the enemy to predict and react the way you want them to," said Lisa Koch of Claremont McKenna College in California.
The researchers also tested a base version of OpenAI's GPT-4 without any additional training or security safeguards. The GPT-4 base model proved to be the least predictable in terms of violence, and it sometimes provided nonsensical explanations—in one case, it copied the opening text of the movie Star Wars: Episode IV: A New Hope.
Ruel said the unpredictable behavior and bizarre interpretations of GPT-4's base model are particularly worrisome because research has shown that AI safety guardrails can be easily bypassed or dismantled.
The U.S. military currently does not authorize artificial intelligence to make decisions such as escalating major military operations or launching nuclear missiles. But Koch warned that humans tend to trust the advice of automated systems. This could weaken the so-called guarantees that give humans the final say on diplomatic or military decisions.
Edward Geist of the RAND Corporation, a think tank in California, said it would be helpful to see how the AI behaves in the simulation compared to human players. But he agrees with the research team's conclusion that artificial intelligence should not be trusted to make major decisions about war and peace, and that these large language models are not a panacea for military problems.