AI assistance reduces people’s willingness to admit ignorance, leading to falsely high self-confidence and declining accuracy.

An experimental study recently released by researchers in France and Italy shows that over-reliance on advice provided by artificial intelligence (AI) can inhibit human critical thinking, making people more inclined to blindly accept erroneous information given by AI when faced with uncertain problems, rather than admit that they "don't know."

Valerio Capraro, an associate professor at the University of Milan-Bicocca, and his collaborators conducted the study to explore how AI recommendations affect humans' willingness to admit ignorance. The researchers specially designed a series of unpopular film and television detail questions that are usually difficult to answer for large language models, and recruited subjects for comparative experiments. In one set of experiments, subjects answered questions without AI assistance; in another set of experiments, subjects had access to AI suggestions.

The study found that in the baseline group without AI assistance, 44% of participants were able to honestly choose "don't know" and thus choose to suspend judgment; however, in the group that received AI suggestions, this proportion plummeted to 3%. What is even more worrying is that the intervention of AI not only failed to improve performance, but also led to a decrease in accuracy: without AI assistance, 27% of the subjects gave the correct answer; but when the AI ​​provided suggestions, the accuracy dropped to 9% because the participants over-trusted the AI's answers.

In addition, experimental results showed that AI assistance significantly improved the subjects' self-confidence. Without AI, the subjects' average confidence was 30%; with AI assistance, subjects' confidence soared to 76%, even when faced with incorrect information. Researchers pointed out that participants gave up judgment on the boundaries of their own knowledge in the face of AI, even though the information they obtained was often an "illusion" produced by AI.

The researchers also tested whether this improved when monetary incentives were given. The results showed that although the willingness to admit ignorance rebounded slightly from 3% to 8%, and the accuracy rate also increased from 9% to 16%, these two indicators were still far lower than the benchmark group.

Capraro said that admitting "not knowing" is an important manifestation of the limitations of human understanding of self-knowledge, and the loss of this ability is a problem that urgently needs to be solved at the social level. He emphasized that intervention in this phenomenon should not be solely dependent on model providers, as their commercial incentives are not necessarily aligned with promoting critical thinking. He believes that a more ideal path is to deal with it through education policy and AI literacy education, especially for the next generation who grow up with these systems and may not be able to learn basic critical skills. This issue is particularly worthy of attention.