OpenAI company recently released data showing that more than one million users discuss the topic of suicide when communicating with ChatGPT every week. According to new data released by the company, about 0.15% of active ChatGPT users have conversations with "clear suicide plans or intentions" every week. Based on the fact that ChatGPT has more than 800 million weekly active users, the number of related conversations has exceeded one million.
OpenAI pointed out that a similar proportion of users show a high emotional dependence on ChatGPT, with "hundreds of thousands" showing signs of psychosis or mania every week. The company emphasized that although such extreme conversations are "extremely rare", they still affect thousands of users every week.
This information disclosure is an initiative by OpenAI to improve AI’s ability to respond to sensitive topics in mental health. The company said that more than 170 mental health experts were consulted during the development of the latest version of ChatGPT. These experts said that the latest version of ChatGPT is "more appropriate and consistent than the old version" in relevant responses. There have been many reports recently on the impact of AI on mental health issues. Some studies have pointed out that AI chatbots may encourage users to have dangerous beliefs and even trigger delusional thinking.
Mental health issues have become a core challenge for OpenAI. Earlier this year, OpenAI filed a lawsuit against the company because the parents of a 16-year-old boy ultimately chose to end his life after communicating suicide-related content with ChatGPT. The attorneys general of California and Delaware also warned OpenAI that it needs to strengthen protection measures for teenage users.
Earlier this month, OpenAI CEO Sam Altman said on social platforms that the company has "been able to alleviate serious mental health issues." However, the relevant specific measures have not been made public. Figures released this week appear to support that commitment and reflect the widespread nature of the problem. Altman also revealed that the company will relax some restrictions, including allowing adult users to communicate with AI for erotic content.
Official data shows that the latest version of GPT-5 is about 65% more likely to achieve desired results in responses to mental health than the previous version; in the evaluation of responses to the topic of suicide, the new version of GPT-5 has a compliance rate of 91%, while the old version is 77%. The new version of GPT-5 has also strengthened its security protection during long conversations, making up for the shortcomings of the old version's reduced protection during long conversations.
In addition, OpenAI has also launched a new evaluation standard that incorporates emotional dependence and non-suicidal mental health crises into AI model safety benchmarks. The company has also launched more parental supervision tools and is developing systems to automatically identify underage users and provide them with stricter security measures.
Although GPT-5 has made improvements in security over its predecessor, OpenAI is still opening older and less secure AI models to millions of paying users. Judging from the current data, some mental health risks still exist, and it is still unknown whether the problem can be completely solved.