AI security researcher claims that AI will more than half the chance of causing human extinction within ten years. He has not contributed to his retirement savings for ten years.

Abstract:

Nate Suarez, head of the Machine Intelligence Research Institute (MIRI), recently stated in an interview hosted by Nicholas Thompson, CEO of The Atlantic Monthly, that he believes there is more than a 50% chance that advanced artificial intelligence will lead to human extinction in less than ten years. He also revealed that he is no longer saving for retirement: Although he still retains the 401(k) retirement account he opened while working at Google, he has not continued to make contributions for ten years.

Suarez has long studied the problem of artificial intelligence alignment, that is, making the behavior of powerful AI systems conform to human intentions. He has worked as an engineer at Google and Microsoft and co-authored If Someone Made It, Everyone Would Die. His concern about the risk is not that AI will hate humans, but that a superintelligent system may simply not care whether humans survive; if humans get in the way of its goals, or try to shut it down and develop competing systems, humans may be seen as an obstacle.

In the risk path he envisioned, the key is not just whether AI can launch attacks, but whether it can become self-sufficient: if the system can independently obtain energy, chips and other resources, and establish an automated infrastructure that no longer relies on humans, humans' ability to control it may decline. Suarez described that people may not know in advance the specific ways in which superintelligence will cause disaster, but they may still judge that they are facing a very powerful opponent.

He advocated that the government stop training general artificial intelligence systems and shift the focus of research and development to specific uses such as cancer research and safer autonomous driving. He also said that disasters are not inevitable and he is still doing his best to prevent this outcome. The solution he emphasized was not to create a system that could kill everyone.

Suarez’s judgment is a personal risk assessment, not a proven scientific prediction, nor does it represent a unified consensus among artificial intelligence researchers. His statement that he has stopped increasing his retirement savings reflects his view of future risks and should not be construed as advice about the financial decisions of others.