According to the Wall Street Journal, AI startup Anthropic called on the world's top AI laboratories to consider slowing down the pace of research and development on the grounds that current AI systems are advancing too fast and may soon be able to improve themselves without human intervention, thus posing major social risks. Anthropic said in a blog post on Thursday that if it could slow down the pace of global AI development, that would "probably be a good thing." The article also disclosed Anthropic’s internal data, showing how quickly the capabilities of its most advanced AI models have improved.
This article was written by Anthropic’s head of internal research and policy. It states,Model development appears to be moving in the direction of "recursive self-improvement," where AI systems can improve themselves without human intervention.Some in the AI industry see this threshold as a sign of potential danger and huge social unrest.
"We believe it would be a good thing if the world could choose to slow down or temporarily pause the development of cutting-edge AI to allow social structure and alignment research to keep pace with technological advances," write authors Marina Favaro and Jack Clark. The article proposes a global agreement on how the development of AI might be slowed, as well as a verification mechanism to verify whether competitors are adhering to the agreement.
The article warns that recursive self-improvement has not yet occurred and is not inevitable, "but it may arrive sooner than most organizations are ready for."
Anthropic's blog
Anthropic recently completed a funding round valuing the company at nearly $1 trillion and has filed confidential documents to begin the process of taking its stock public. The company has recently emerged as a front-runner in the fierce competition with OpenAI for AI supremacy. OpenAI is also expected to submit IPO-related documents soon.
The company has emphasized AI safety since its inception, but has long been criticized by the outside world, believing that its policy proposition is actually an attempt to slow down the progress of competitors' AI research and development. David Sacks, a venture capitalist and informal adviser to President Trump, has accused Anthropic’s leadership of pursuing a “regulatory capture agenda” that uses regulation to suppress competitors.
In a recent podcast, Sacks said this "regulatory capture agenda" in Washington could ultimately lead to a ban on the so-called open source model. These models are typically less expensive and easier for businesses and institutions to deploy and develop on their own.
Others believe that Anthropic's warnings about the dangerous potential of its own tools could also be seen as a marketing ploy.These skeptics point to Anthropic's decision to limit the release of its powerful "Mythos" cybersecurity model, which can find vulnerabilities and issues in code, as a clever promotional ploy to tout its product's capabilities.
In this regard, Anthropic’s leadership stated that the company takes AI security issues seriously and is committed to promoting more social discussion on related risks.