A bill before the New York State Legislature would ban artificial intelligence chatbots from providing legal or medical advice to the public and give users the right to sue owners of offending chatbots. The bill, numbered S7263, was introduced by the state Senate during the last legislative session. It was approved out of the committee by a vote of 6 in favor and 0 against in the Internet and Technology Committee last week. It is one of the legislative packages to regulate AI chatbots.
The bill focuses on AI chatbots that pretend to be licensed professionals (such as doctors and lawyers), prohibiting them from providing "substantial responses, information or advice" that may violate professional licensing laws or constitute "unlicensed practice."
The bill requires owners of chatbots to provide users with "clear, prominent and unambiguous" prompts that they are interacting with an AI system. The prompts must be in the same language as the chatbot and presented in a readable font. However, the bill also emphasizes that this "non-human system" prompt does not exempt the chatbot owner from legal liability. Under this framework, if the operator violates the ban, users can take the chatbot owner to court based on the civil litigation rights granted by the bill, claiming damages and requesting payment of attorney fees. Some experts believe that similar private right of action clauses are important enforcement tools in data and AI regulatory laws. The lack of this mechanism often weakens legal deterrence. The Attorney General of Maine in the United States recently called it a "significant deterrent effect" when discussing data privacy legislation.
If the bill is finally passed and signed by the governor, it will be officially implemented 90 days after it takes effect. The legislative portfolio in which S7263 is included also includes a number of measures targeting AI and the protection of minors, such as restricting the use of features in chatbots that are unsafe for minors and regulating certain online platforms (including the gaming platform Roblox) that are considered weak in protecting the privacy of minors. Other items in the package of legislation involve requiring generative AI systems to provide users with instructional prompts for use, as well as establishing new rules on the processing of biometric data and "synthetic content creation."
Senator Kristen Gonzalez, the sponsor of the bill and chair of the State Senate Internet and Technology Committee, said that this package of bills aims to ensure that AI innovation "does not come at the expense of the safety of New Yorkers, especially children." In January this year, Character.AI, a free generative AI chat app, reached a settlement with Google in several lawsuits related to minor suicides. The focus of the case was the role these chatbots played in the tragedy. Such incidents also pushed states to speed up the regulation of chatbots. Gonzalez said in a press release that the public "deserves real care from real people" as well as transparency, accountability and a commitment that their data is securely protected when using technology.
Currently, these New York state-level initiatives are also placed in the context of a broader national AI regulatory game, involving multiple trade-offs between state and federal, privacy and innovation, security and openness. After this case, issues such as whether chatbots can provide professional advice, how to remind them of their "non-human status", and who will be responsible if they cause harm are expected to continue to ferment in legislation and judicial practice at all levels in the United States.