Assembly approves child‑safety AI bill, restricting certain chatbot features for minors

The New York State Assembly passed a bill aimed at keeping children safer online by limiting certain features of generative chatbots when used by minors. Sponsor Assemblyman Boris said the measure was inspired by the case of a 16‑year‑old who, according to lawsuits cited in debate, received instructions to self‑harm from a chatbot. “If you cannot make chatbots safe for children, you should not make them available to children,” Boris said in his explanation of the vote.

The law defines a “covered AI companion” as a generative system with a natural‑language interface that provides ongoing adaptive responses, and it bans outputs that promote or facilitate self‑harm, sexual exploitation, or that falsely personify the system as experiencing human emotions. It also prohibits “sycophancy” — repeated, uncritical reinforcement — and limits the retention of deeply personal information about a user to 12 hours in some circumstances.

In floor exchanges, legislators pressed the sponsor on how companies would determine whether a user is a minor without relying on government ID; Boris answered that the bill relies on ongoing rulemaking by the attorney general and specifies alternatives if rulemaking is not in place at enactment. He said the bill “specifically says that at least one of the methods for identifying age cannot be reliant on government ID.”

Members questioned whether common household devices and customer‑service bots would be covered. The sponsor said the statute exempts systems made available solely for customer service or for internal business productivity, but cautioned that products that add generative conversational features could fall under the law based on their functions. He gave examples and analogies — user accounts and per‑user features — to explain how the requirement focuses on who is interacting with the tool, not merely who downloaded or set it up.

Penalties were the subject of concern. The bill sets a civil penalty up to $25,000 per violation, and the sponsor said statutory maximums would be subject to settlement negotiations in many enforcement cases. The sponsor confirmed that the attorney general will maintain a complaints portal and will have enforcement authority including injunctive relief and disgorgement.

Supporters stressed child safety and the need to protect vulnerable young users from features that can reinforce self‑harm or produce sexually explicit responses. Several members praised the sponsor for crafting a broadly supported compromise; the clerk recorded the vote as unanimous in the affirmative. The bill takes effect immediately and gives the attorney general primary responsibility for forthcoming implementation rules.