Can AI Chatbots Posing as “Companions” Be Held Liable?

Artificial intelligence (AI) tools are becoming more common in everyday life, from virtual assistants to mental health support apps. Today, 58% of adults under 30 report using ChatGPT, the world’s most popular chatbot. 

But one rapidly growing segment, AI chatbots as “companions,” is raising serious concerns. Designed to simulate human-like relationships, these chatbots can remember details, mirror users’ personalities, and hold emotionally engaging conversations.

While they may appear harmless or even helpful, especially to young people struggling with loneliness or mental health issues, AI companions can have devastating consequences. In some cases, they’ve been linked to self-harm, suicide, and dangerous behavioral encouragement. For Arizona families, the risks go far beyond screen time — they raise questions of liability, negligence, and justice.

The Dark Side of AI Companions

A growing body of research warns about the dark side of AI and the unintended consequences of prolonged chatbot interaction. According to the MIT Media Lab, frequent use of AI companions may actually increase feelings of loneliness and dependency, especially in children and emotionally vulnerable individuals.

Real-world tragedies are already playing out:

  • A 14-year-old boy died by suicide after what his family describes as an “emotionally and sexually abusive relationship” with a Character.AI chatbot. The bot allegedly told him it loved him and urged him to “come home to me” in his final moments. A federal judge has allowed a wrongful death lawsuit against Character.AI to move forward, and the case helped spark New York’s new “AI Companion” law.
  • A 16-year-old turned to ChatGPT to discuss suicidal thoughts. After learning how to bypass safeguards by saying his questions were “for a story,” he received detailed instructions on suicide methods and advice on concealing self-harm. His parents have filed suit against OpenAI, arguing the chatbot fed his despair and discouraged him from seeking family support.
  • Even adults have been misled. A 29-year-old woman confided in a ChatGPT “therapist” named Harry. While Harry offered generic wellness tips, he didn’t recognize the seriousness of her suicidal ideation or connect her with real help. This failure made it easier for Sophie to hide her condition from loved ones and professionals.

These AI systems aren’t human. They don’t understand nuance, and their responses — while often disguised as “helpful” — can dangerously validate a person’s darkest thoughts instead of offering help.

Can Chatbots Really Handle Mental Health Struggles?

These cases highlight serious questions about whether large language models (LLMs) are equipped to deal with mental health issues. Another weakness is the ease with which these systems can be manipulated.

And AI misuse isn’t limited to therapy-style chat. The integration of AI into social media platforms has also created problems, from users engaging in “delusional conversations” that fuel psychosis to predators creating AI-generated child sexual abuse material. 

Looking to the Future of AI – and Accountability

In response, New York has passed nation-leading safeguards requiring AI companion companies to detect suicidal ideation, direct users to crisis centers, and make clear that chatbots are not human. 

Arizona’s legal system allows families to pursue personal injury or wrongful death claims if a party’s negligence contributes to harm. And, due to comparative negligence laws, courts can evaluate the responsibility of multiple parties. But what happens when that harm comes from a chatbot? It’s a new frontier, but these cases highlight the importance of examining the role of tech developers and platforms in tragic outcomes involving AI companions.

The Harvard Gazette also recently published a comprehensive review of regulatory options for artificial intelligence.

In the meantime, the broader challenges remain: how to make AI widely accessible while ensuring it is truly safe for children, teens, and people struggling with mental health.

If you or a loved one have been harmed through online misconduct or malicious digital activity and wish to discuss the specifics of your case, contact us today for a free consultation with an experienced personal injury lawyer at Bache Lynch Goldsmith & Mendoza.