Abstract:
OpenAI recently announced that it will strengthen ChatGPT’s restrictions on violence-related content in the next few weeks, adding protection for weapons and security system information for minor users. According to the new measures, ChatGPT can end the conversation on its own when the conversation shows that the user continues to focus on violence or the idea of seriously harming others. If the conversation is ended multiple times due to such issues, the account may even be banned.
ChatGPT's original policy prohibits providing guidance to help users plan or carry out violent acts. This adjustment will limit the details provided by the model when answering questions about "past large-scale attacks and perpetrators", that is, the model will reduce the content or even refuse to answer from a security perspective.
The scope of the restrictions includes recreating the weapons, equipment and acquisition methods involved in the attack, as well as limiting user queries for detailed comparisons of the scale of damage that may increase the attacker's visibility. The above restrictions apply to all users, regardless of age group or purpose of asking questions, so even adult users will face the same restrictions.
For users identified as under 18, OpenAI will also limit useful information about weapons and security systems that may be used to commit violence. This protection will also apply to problems that on the surface do not appear to indicate violent intent. OpenAI says teens can still use ChatGPT to get help, learn how to prevent or de-escalate violence, and connect with trusted adults.
The improved age prediction mechanism will combine various signals such as user behavior to continuously evaluate whether the account may belong to a minor. If appropriate, the system will automatically apply relevant protection measures for users under the age of 18. If a user is mistakenly identified as a minor, they can also actively submit their identity information for verification.
Comments