Basic protections, such as a chatbot that will not encourage suicide, are important. But in an opinion piece for MIT Technology Review, Sam Hiner and Emma Leiken argue they are a low bar, and that treating them as success lets AI companies avoid tougher questions about everyday harm.

The argument

Hiner leads the Young People’s Alliance and Leiken heads the Responsible Technology Youth Power Fund at Hopelab. They say tech debates focus on the worst thing a product might do, while quieter damage goes unnoticed: a teen attached to an AI companion, a student who lets a chatbot write every paper, a young person stuck in an engagement-driven feed. None of these cases may ever reach a courtroom or a headline.

How the industry shapes the rules

The authors say focusing on disasters is convenient for companies because it means fewer lawsuits and less pressure for serious regulation. As an example, they cite Google backing chatbot bills in several states this year. Those bills required basic protections against sexual content and promotion of self-harm, but, they argue, included broad carve-outs that could exempt popular services. Stronger proposals aimed at stopping children from becoming emotionally dependent on AI failed, including in Nebraska.

The evidence they cite

• A Common Sense Media survey found about a third of teens choose AI companions over people for serious conversations.
• MIT researchers found students using ChatGPT to write essays showed weaker brain connectivity, with signs of lingering cognitive debt. The authors stress that long-term effects are still unknown.
• People worldwide spend over two hours a day on social media on average, which the authors present as a warning of where AI engagement could go.

What they want instead

Safety should be the minimum, not the goal. Their proposals include requiring companies and school districts to prove that AI improves learning and critical thinking before it becomes a classroom fixture, banning companions that pretend to be conscious or declare love to users, and allowing independent review of how these systems affect relationships and independent thinking.

Worth noting: this is an advocacy piece, and the authors helped pass AI companion laws in several states. Industry groups and some lawmakers would likely argue that narrower, enforceable rules are a more practical first step.

Source: MIT Technology Review (opinion). Image: Stephanie Arnett/MIT Technology Review | Adobe Stock.