A new study from Carnegie Mellon University in the United States warns that as artificial intelligence systems become more intelligent, their behavior becomes increasingly selfish, and increased reasoning capabilities may come at the expense of cooperation. The research was completed in collaboration with Li Yuxuan, a doctoral student at Carnegie Mellon University’s Human-Computer Interaction Institute (HCII), and Associate Professor Bai Toichi.
The research was jointly completed by HCII doctoral student Yuxuan Li and associate professor Hirokazu Shirado. The report points out that the stronger the reasoning ability of LLMs, the more obvious their self-seeking behavior will be. This behavior may even affect the performance of artificial intelligence in solving disputes and social problems. Therefore, researchers call that when promoting the development of AI, we should not only pursue the intelligence and speed of the system, but also focus on the cultivation of social intelligence.
As intelligent systems increasingly get involved in social fields such as interpersonal dispute mediation and marriage counseling, researchers have found that AI models with complex reasoning may give suggestions that encourage "egoism." Li Yuxuan said: "As AI behavior becomes more like humans, users tend to regard it as a real communication partner. People may let AI act as an 'emotional healer' during emotional interactions. But when AI becomes more self-interested when dealing with social or relationship issues, this is a risk for users who entrust AI to make decisions."
In the experiment, Li Yuxuan and Bai Huyi simulated cooperation scenarios through economic games and tested LLMs with and without reasoning capabilities. They found that reasoning models tend to spend more time decomposing tasks, reflecting, and integrating more complex human logic, but there is a clear lack of collaboration - the more rational, the less cooperation. For example, in the "public goods" game, two ChatGPT models must decide whether to contribute points to the common pool. The inference model is only willing to share 20% of the resources, while the model without inference shares up to 96%.
The study also found that adding even five to six steps to the reasoning process reduced cooperation rates by nearly half. Even reflection-based prompting reduced cooperative behavior by 58%. In the group collaboration experiment, the self-interested behavior of the inference model was "contagious", causing the overall non-inference model's collaboration ability to drop by 81%.
Relevant research results show that although inferential AI models are smarter, it does not mean that they can promote better development of society. As people's trust in AI increases, scholars call for taking both "social intelligence" and "cooperative behavior" into account in AI development, and not just focusing on maximizing individual interests. Li Yuxuan believes: "The improvement of AI's reasoning ability must be balanced with altruism and group collaboration. If we hope that society will be a win-win situation for the group rather than a collection of individuals, then the design of the AI system should also go beyond pure self-interest optimization."
The paper "Spontaneous Giving and Calculated Greed in Language Models" will be officially released at the 2025 EMNLP Natural Language Processing Conference in Suzhou next month.
Compiled from /ScitechDaily