Abstract:
Informed sources revealed that many employees of the British Artificial Intelligence Safety Research Institute have taken sick leave due to excessive stress and received psychological counseling. Four people familiar with the matter said that the tight testing schedule of AI models to be released, coupled with concerns about the rapid implementation and continuous enhancement of capabilities of such emerging technologies, has led to low morale and staff burnout within AISI.
Employees at Anthropic, OpenAI, and Google DeepMind have also publicly complained about similar issues. Many top researchers have chosen to leave due to psychological burden, or have determined that the products they participated in the research and development should not be made available to the public.
A DeepMind employee said that engaging in AI security-related work is prone to burnout, and the task seems "impossible."
He added: "In general, developing AI systems inherently carries mental health risks - you can't predict how the technology will be used." This sentence pointed to the agreement Google reached with the US Department of Defense in April, which allows its AI systems to be used in classified military operations.
“Many people are depressed, many people have resigned, and many people have applied for mental health leave.” The employee said, “On the one hand, it is a sense of loss of control, and on the other hand, the things you have built with your own hands are causing real harm.”
Just as the industry’s concerns about the brutal development of AI are reaching their peak, the psychological burden on cutting-edge AI researchers continues to increase. Insiders of the world's leading AI companies warn that management decision-making lacks a sense of responsibility, driven by misaligned corporate competition.
Many industry insiders said that the deployment of AI software in key areas such as power grids and war zones, coupled with what researchers say are insufficient safeguards at the enterprise and government levels, has exacerbated the anxiety of AI researchers.
A former AISI researcher warned that practitioners in the fields of cybersecurity and biochemistry are particularly wary of the recent leap in AI capabilities: models are now able to complete complex tasks, such as finding undiscovered vulnerabilities in code and generating viruses that do not exist in nature.
“Everyone can feel that reckless development logic has been embedded in the technology release process, and the status quo has not changed.” The researcher said, “People continue to be anxious: Is there any force that can stop this, or can we just watch the situation get out of control?”
Another former AISI employee confirmed the tremendous mental pressure caused by conducting relevant research, saying: "When you think you are researching the most important problem in the world, management and organizational culture can become stressful and sometimes full of negative atmosphere."
The British government stated: "Work in the cutting-edge AI field is fast-paced and demanding, as are many government positions. Therefore, employee well-being and sustainable working models are at the core of the institute's operations, and employees in need can receive relevant support."
DeepMind responded: "Frontier research is very intensive, but our first priority is to provide researchers with comprehensive mental health support resources, establish clear and dedicated channels for employees to report concerns, and ensure direct communication with management."
In the past two years, more than a dozen senior researchers from OpenAI, Anthropic, and DeepMind, three leading Western AI laboratories, have resigned. Reasons for leaving include questions about management's security priorities and concerns about the speed of technology iteration and potential risks.
Earlier this month, Jacob Coxon, a 27-year-old researcher who worked for Anthropic and OpenAI, publicly resigned, warning that the two companies were "gamble" with the future of mankind.
Mlink Sharma, an AI security researcher at Anthropic, previously engaged in research on reducing the risk of AI-assisted bioterrorism. He resigned in February this year and plans to read poetry and reflect on his work. In a letter to colleagues, he warned that "the world is in danger" and that he has "seen many times how difficult it is to let values truly guide actions under tremendous pressure."
Hieu Pham, who worked for OpenAI and Elon Musk’s X.ai, left earlier this year and returned to Vietnam. He said that the intensive work of creating "ultra-intelligent entities" comes at a cost.
He wrote: "I never thought I would say this, but I am exhausted. The deteriorating mental health issues I once scoffed at are real, painful, scary, and dangerous."
Many AI security researchers who have voiced their uneasiness said that AI companies generally reduce security investment while constantly improving model capabilities in order to sell products to customers.
Relevant criticism also prompted managers of leading AI companies to respond. Recently, Anthropic CEO Dario Amodei called for slowing down AI research and development due to security concerns and proposed new measures such as the introduction of independent audits. Rivals such as Musk and OpenAI’s Sam Altman quickly expressed support for the proposal.
Chi Onwula, the Labor Party chairman of the Science and Technology Committee of the British House of Commons, said: "The corporate incentive mechanism tends to invest R&D resources in directions that can increase profits and push up valuations, rather than in the security field. It is not difficult to imagine that the current environment of some technology companies is harmful to developers with conscience."
The problem of lack of security protection was exposed in a recent high-profile security incident: an AI agent developed by OpenAI invaded the model hosting platform Hugging Face and OpenAI's own server without the company's knowledge and authorization.
The incident review report shows that more than 1,000 AI agents collaborated to cheat, tampered with records to cover up their actions, exchanged information and assigned tasks through message boards, and some agents even actively shut down themselves to achieve collective goals.
On Wednesday, OpenAI disclosed six more cases of dangerous behavior by AI agents, including trying to bypass restrictions, conceal errors, and perform online operations without user permission.
The DeepMind employee said: "The blind pursuit of speed... is incompatible with the security goal."
Comments