Abstract:
Jacob Coxon announced his resignation from Anthropic on X. He said that he has been engaged in pre-training research at OpenAI and Anthropic for the past three years, but believes that neither company is advancing AI development in a responsible manner. Instead, they are "going straight to self-improving superintelligence" and putting human safety at risk.
Coxon said that the AI systems currently being built in these closed laboratories will soon have "superhuman" capabilities, able to carry out hacking attacks, change the pattern of a certain field overnight, and gain power and resources in the real world. He pointed out that relevant research progress has continued to advance in multiple directions and shows no obvious signs of slowing down.
He also mentioned that judging from the public statements of OpenAI CEO Sam Altman and Anthropic CEO Dario Amodei, the outside world may think that both companies are cautiously advancing the development of AI, but he believes that this is not the case. Coxon claims that these practitioners privately believe that AI "could kill us all by the end of the century," and that this is not marketing talk; he also said that executives such as Altman and Amodei will adjust their rhetoric when facing the media to appear more restrained, but privately they are actually deeply worried about what is being created.
In his opinion, the internal cultures of OpenAI and Anthropic are also different. Coxon said that many employees at OpenAI are not "deeply aware of the civilization-level risks involved"; at Anthropic, the relevant risks are "very clear to everyone", but the company is still locked in the race to be the first to develop AGI. Coxon said Anthropic believed that since others might not act responsibly, it had to win the race first.
Coxon believes that in this context, many key decisions should not be completed only in Slack in private companies. He is cautiously optimistic about increased coordination among U.S. AI labs after recent events, but believes that such cooperation will be ineffective once it is extended to laboratories outside the United States. To this end, he proposed that a temporary ban on model capacity improvements could be considered.
At the end of the post, Jacob Coxon called on other laboratory researchers to seriously think about the next five years, not to continue to avoid problems, but to push AI development in different directions.
Comments