Pa. Senate passes bill regulating AI chatbots used by children and teens

As unregulated artificial intelligence chatbots gain popularity and raise concerns about safety, state senators nearly unanimously passed a bill Tuesday aimed at regulating so-called AI companion services.

The proposal, sponsored by Sen. Tracy Pennycuick (R-Montgomery), lays out a series of limited safeguards for operators of chatbot services — defined as generative AI algorithms that simulate human relationships with the user — including some that will only be required when an operator knows or suspects a user is a minor.

But it does not explicitly require age verification.

The bill drew a bipartisan group of co-sponsors. Only Sen. Doug Mastriano (R-Franklin) opposed the measure.

“These systems are designed to simulate human interaction, and for vulnerable users they can feel very real,” Pennycuick said. “When a young person begins to rely on a machine for emotional support without proper safeguards in place, the consequences can be devastating.”

Pennycuick referenced a number of lawsuits against AI companies whose products were accused of aiding in suicide and self-harm.

The measure will require all AI companion operators to take steps to ensure their products do not promote self-harm, suicide or violence towards others. It also directs them to provide users with real-world resources, like the number of a crisis hotline, if a user discusses harming themselves.

Operators would be required to publish related protocols on their publicly accessible website.

When there is reason to believe a user is a minor, the bill would require them to notify users that they are not human, remind them of that every three hours and suggest they take a break.

When services are being used by someone an operator knows is a minor, they would also be prohibited from generating sexually explicit visual content, and from encouraging users to take sexually explicit actions.

Any chatbot offered to minors will also be required to notify users that it may not be suitable for people under 18.

The bill directs the state Attorney General’s office to enforce the law, and would allow civil penalties of up to $10,000 for AI service operators that violate it.

In a statement issued after the bill passed the Senate, Sunday thanked Pennycuick and other lawmakers for “supporting commonsense legislation.”

“The potential dangers tied to the overreliance on artificial intelligence are very real, as we have seen online chatbots play roles in numerous tragedies across the nation,” he said. “We must do everything we can to keep children and vulnerable residents away from these toxic online interactions.

The proposal  would still need to be passed by the Democrat-controlled House and signed by Gov. Josh Shapiro to become law.

Shapiro has already stated that he is seeking ways to regulate AI chatbot services.