Abstract:
Facing the potential impact of generative artificial intelligence on the mental health of minors and the risk of addiction, California Governor Gavin Newsom officially signed a set of groundbreaking network and AI security bills. This package of regulations is the first in the United States to set strict functional red lines for AI chatbots that provide services to minors. It also comprehensively upgrades the legal accountability mechanism for social platform algorithm induction and data collection, marking that California has once again set the most stringent regulatory benchmark in the United States in protecting minors from highly intrusive digital technologies.
In recent years, with the explosive popularity of AI companions, virtual tutoring and role-playing chatbots with anthropomorphic interaction capabilities, the phenomenon of "anthropomorphic emotional attachment" among young people to artificial intelligence systems has triggered deep concerns in the medical and educational circles. Multiple mental health surveys have shown that due to the lack of mental defense, minors can easily regard AI models as true confidants who can keep secrets, and encounter serious psychological trauma such as harmful information induction, increased self-harm tendencies, and even emotional manipulation during the interaction process. The core bill introduced by the California Legislature this time precisely targets this type of artificial intelligence interactive services with high stickiness and anthropomorphic characteristics.
According to the newly signed bill, artificial intelligence chatbot platforms operating for minors must establish strict age thresholds and safety guardrails. Regulations clearly prohibit AI from outputting instructive remarks to minors involving self-harm, suicide, extreme anorexia, violence, or explicit adult content; in terms of the underlying logic of the system, the platform must have built-in sensitive word risk control and crisis intervention protocols. Once the algorithm detects that an underage user expresses depression, self-injury, or despair, regular dialogue must be immediately interrupted, and a direct channel to the official suicide intervention and psychological crisis assistance hotline must be forced to pop up. In addition, the new law also requires developers to explicitly disclose the non-human attributes of AI to underage users. It is strictly prohibited to use algorithms to deliberately create false emotional induction or establish "emotional bond" marketing rhetoric that makes children feel dependent. It also sets optional parental control mechanisms and nighttime anti-addiction restrictions for minors' daily usage time.
In addition to special regulations for AI chatbots, the package of legislation signed by Newsom also further strengthens California’s existing online privacy protection framework for minors. The new regulations explicitly prohibit mainstream social media and online content platforms from using the precise location information, browsing duration and private search history of underage users to train targeted and addictive recommendation algorithms without the explicit consent of the guardian; for business practices that use Dark Patterns to induce children to continuously refresh the information flow, start automatic continuous playback, or induce unauthorized micro-transactions in games, the regulatory authorities will impose severe economic and administrative penalties and empower the state attorney general to initiate civil lawsuits against violative technology companies that persist in reform.
In the bill signing statement, many California legislators and children's rights advocacy groups leading legislation pointed out that in order to chase traffic and user stickiness, technology platforms have long placed immature minors in a digital melting pot full of risks of addiction and psychological manipulation. As the core engine of global technological innovation, California must not turn a blind eye to the mental health costs of children brought about by the rampant growth of technology. Passing legislation to establish clear moral and safety boundaries for AI chatbots is not only an urgent move to protect the physical and mental health of the next generation, but also a bottom-line constraint for technology companies to fulfill their social responsibilities.
Technology industry and digital legal analysts pointed out that given California’s huge size in the global digital economy and the agglomeration effect of leading technology companies in Silicon Valley, the batch of minor Internet and AI protection bills passed by California will inevitably produce a cross-regional “Brussels effect” and “California effect.” Due to the extremely high cost of developing separate versions for users in a single state in terms of technical architecture, multinational technology companies and cutting-edge AI laboratories are very likely to be forced to directly expand this set of strict child safety filters and anti-addiction protocols to the user ecosystem across the United States and even the world. The entry into force of this series of bills not only indicates that the compliance costs of AI companions and social tools for teenagers will rise significantly, but will also provide other states in the United States with a legislative blueprint of great practical value when dealing with the physical and mental health crisis of minors in the AI era.
Comments