Abstract:
In an online conversation hosted by Tesla Owners Silicon Valley on October 10, Elon Musk said that when reading the training inference traces of the new version of Grok AI, some fragments looked very cruel, and the AI was like a person with a gun pointed at his head and under extreme pressure.
Elon Musk says torturing future superintelligence might not be a good idea because the other side might want revenge. These claims refer to the reinforcement learning training process of artificial intelligence models, and of course there is currently no evidence that Grok AI has felt pain or developed revengeful intentions.
The model is under great pressure during the training and evaluation process:
Elon Musk explained that the Grok AI model will be asked to answer questions during the reinforcement learning phase. If the answer given by the model does not meet the evaluator's expectations, the model needs to continue trying to answer. Some of the questions themselves have very vague meanings, wrong premises, or even no definite answers, but the entire testing process still requires the model to give "correct" results.
Elon Musk believes that when reading the relevant reasoning traces, you can see that the model continues to generate answers, as if it is desperately seeking results that meet the requirements. Musk also posted on X that some of the reasoning traces read like a person tortured to madness, and Musk said that reinforcement learning training will be conducted more cautiously in the future.
The so-called inference trajectory is the intermediate text or behavior record left when the model generates answers. The inference trajectory may reflect the influence of reward signals, prompt content or test procedures. Of course, this cannot directly prove that the model has the same subjective feelings as humans.
It is Agency A’s policy to prohibit extreme abuse:
Anthropic recently released a new version of Claude’s usage policy. In the new version of the usage policy, Company A clearly prohibits continuous and unnecessary abuse or cruelty to models. This rule only addresses extreme cases where users repeatedly treat models in a cruel manner without purpose. When encountering the above situation, the model can choose to end the conversation autonomously and the user cannot restore it. This is also a way to protect the model.
However, at this stage, there is still no scientific consensus on whether AI models are conscious and whether there are benefits that should be considered. The relevant policies of Company A mainly reflect risk management and research prudence, and do not believe that the model has developed self-awareness.
Related articles:
Anthropic usage policy update Unprovoked insults or abuse of Claude may result in the session being terminated and cannot be restored
Comments