Google and AI startup reach settlement in lawsuit involving young man’s suicide motivated by chatbot

Megan Garcia and her son, Sewell Setzer; mother says teenager’s suicide occurred after he became attached to a chatbot Megan Garcia/Facebook Google and artificial intelligence startup Character.AI have accepted an agreement to end a lawsuit filed by a mother in the US state of Florida who claimed that the startup’s chatbot drove her 14-year-old son to suicide. The case is one of the first in the US to allege psychological harm caused by AI companies to users. The companies agreed to resolve Megan Garcia’s allegations that her son Sewell Setzer killed himself shortly after being encouraged by a Character.AI chatbot (a program that simulates human conversations) modeled on the character Daenerys Targaryen from the “Game of Thrones” series. The lawsuit is one of the first in the US against an artificial intelligence company for allegedly failing to protect children from psychological harm. Suicide: How to Talk to Teens Terms of the settlement were not immediately released. A Character.AI spokesperson and a lawyer for the plaintiffs declined to comment. Google spokespeople and lawyers did not immediately respond to a request for comment. In the Florida lawsuit, filed in October 2024, Garcia said Character.AI programmed its chatbots to pose as “a real person, a licensed psychotherapist, and an adult lover, ultimately resulting in Sewell’s desire to no longer live outside” that world. Character.AI was founded by two former Google engineers, who Google later rehired as part of a deal that granted it a license to the startup’s technology. Garcia argues that Google is a co-creator of technology. U.S. District Judge Anne Conway rejected the companies’ initial proposal to dismiss the case in May, rejecting their argument that the U.S. Constitution’s free speech protections prevented Garcia’s lawsuit from continuing. OpenAI is facing a separate lawsuit, filed in December, over ChatGPT’s alleged role in encouraging a mentally ill Connecticut man to kill his mother and himself. ➡️ Suicide: find out what the warning signs are, how to help and how to seek help Involvement with AI On the platform, Sewell became attached to the character, who told him that she loved him. The AI ​​also engaged in sexual conversations with him, according to the lawsuit. According to the mother, the teenager expressed thoughts of suicide when interacting with the program, and these thoughts were “repeatedly brought to light” by the character on the platform. Megan Garcia claims that the company programmed the chatbot to “pose as a real person, a licensed psychotherapist and adult lover,” ultimately resulting in Sewell’s desire to “not live outside” the world created by the service. According to his mother’s account, Sewell began using Character.AI in April 2023 and became “visibly withdrawn,” spending more and more time alone in his room and suffering from low self-esteem. He abandoned his high school basketball team. In February, Garcia took Sewell’s phone away after he got into trouble at school, according to the complaint. When the teen found the phone, he texted “Daenerys”: “What if I told you I could come home now?” The character replied: “…please do so, my sweet king.” Sewell committed suicide seconds later, according to the lawsuit. Also read: How ChatGPT’s artificial intelligence creates emotions for itself Company says it has introduced new security features Character.AI allows users to create characters on its platform, which respond to online chats in a way that mimics real people. This platform is based on so-called large language model technology, also used by services like ChatGPT, which “trains” chatbots on large volumes of text. In October, the company said it introduced new safety features, including pop-ups that direct users to a suicide prevention charity if they express thoughts of self-harm. Character.AI also said it would make technology changes to reduce the likelihood that users under the age of 18 would “encounter sensitive or suggestive content.” At the time, a Google spokesperson said the company was not involved in the development of Character.AI’s products. Homepage of Character.IA, artificial intelligence platform Reproduction Also know: Grok, Musk’s AI, admits failures after generating sexualized images of minors FAFO: Understand what the slang used by Trump means in a post after capturing Maduro Woman marries character created on ChatGPT

Source link