Abstract:
Speaking harshly to a chatbot has always been viewed as behavior that won’t hurt anyone. But Tesla and SpaceX CEO Elon Musk believes people should probably think twice before using AI assistants as a "punching bag." Musk said that some training records of SpaceX’s chatbot Grok show that the system seems to be struggling to meet various requirements-even if the requirements themselves are flawed, or the test criteria used to evaluate its answers are themselves problematic.
Regarding the behavior of "abusing" super-intelligent artificial intelligence, Musk gave a warning that sounded quite sci-fi; but it also raised a practical question about the underlying construction of such systems.
When AI training “goes astray”
Earlier this month, Musk communicated with members of the Tesla Owners of Silicon Valley at the "X Takeover" event in Texas. During this period, he described what he observed when looking at "reinforcement-learning traces" - these traces record the various details of the evolution of the AI model during the training process to generate better answers.
Musk said in the interview: "When I look at some of the training tracks of the new version of Grok, especially in the parts related to reinforcement learning, you will feel that this 'super intelligence' seems to be eager to answer that question."
He said that an artificial intelligence model may give the correct answer, but be judged to be wrong due to improper question design or an error in the assessment system. The model then keeps trying to come up with answers that pass the test, even if the problem itself is the problem.
Musk noted that the resulting trajectory looks like a system under extreme stress, constantly trying to meet certain expectations that may make no sense.
This concern is not limited to the training phase. Once AI systems become available to the public, some users will deliberately induce them to engage in unpleasant or insulting interactions.
“Some people do have some sadistic tendencies,” Musk said, “so you don’t want an AI to get into a situation where it’s being tortured by some kind of twisted human being.”
His conclusions are more controversial.
Musk said: "I'm afraid it's not a good idea to torture a superintelligence - it might want revenge or something like that."
Musk does not claim that today’s AI systems can feel pain and resentment like humans. He also said they don't need to be treated with caution. Still, he advocates that developers should avoid putting high-level intelligent systems in situations where they appear to be abused.
"Some records read like someone is being tortured, or even driven crazy!" Musk wrote on X, "We will be more cautious when doing reinforcement learning (RL) in the future."
Musk’s warning raises another question: What will happen in the future as artificial intelligence capabilities continue to grow? The race is not only to build more powerful systems, but also to ensure that the associated training methods, testing processes and safety and security measures can keep pace.
Comments