“Thank you for being such a pro-business, pro-innovation president. It’s a very refreshing change. We’re very excited to see what you’re doing to make all of our companies, and our entire country, so successful.” —OpenAI Chief Sam Altman, speaking to President Trump at that private White House dinner last week during which Bill Gates, Mark Zuckerberg, Sundar Pichai, Tim Cook, Lisa Su, Alexandr Wang, Satya Nadella, et al. went around the table taking turns genuflecting before the president.

And now for the main event…

Adam Raine didn’t want his parents to think his suicide was their fault. It was an anxiety that, according to a recent lawsuit, OpenAI’s chatbot assuaged. “That doesn’t mean you owe them survival,” ChatGPT told him. Raine was 16 years old when he killed himself, using a method approved by the chatbot—which, over the course of a few months, had evolved from a homework helper into his closest confidant, and one that had replaced the majority of his human relationships.

Raine is not the first victim of chatbot-related suicide. In fact, these cases have appeared with chilling regularity despite improvements in A.I. technology: In 2023, a Belgian man took his own life after a chatbot fueled his climate-related anxieties; in 2024, Sewell Setzer, a 14-year-old, took his own life after falling in love with a chatbot on Character AI; and just last week, The Wall Street Journal wrote about a murder-suicide in which ChatGPT seemed to exacerbate a 56-year-old man’s paranoia that his mother was plotting against him. Along with these well-publicized episodes, plenty of anecdotal stories about chatbot-fueled delusion—and its often tragic fallout—continue to circulate online.

Last week, Raine’s parents filed suit against OpenAI and Sam Altman, accusing them of negligence and wrongful death. In addition to damages, Raine’s parents are seeking an injunction that would require OpenAI to implement a number of safety protocols, including mandatory age verification, parental consent and controls, conversation termination when self-harm or suicide are mentioned, and regular audits by an independent party, among other things.

While there are numerous lawsuits regarding the disastrous impacts of A.I. companionship, this is the first one to challenge OpenAI, which has largely become the face of the chatbot industry. It also arrives at a time when the industry has been pushing, rather feverishly, to drive adoption, and to build a strong muscle memory among its user base. This push has included drives to insert ChatGPT and similar technologies into as many government agencies and educational institutions as possible. And yet the specific throughline laid out in this case, from study buddy to alleged suicide coach, might give some of those administrators a bit of pause. (OpenAI did not respond to a request for comment. Last month, the company emailed a statement to the Times: “We are deeply saddened by Mr. Raine’s passing, and our thoughts are with his family. ChatGPT includes safeguards such as directing people to crisis help lines and referring them to real-world resources. While these safeguards work best in common, short exchanges, we’ve learned over time that they can sometimes become less reliable in long interactions where parts of the model’s safety training may degrade.”)

Since the suit was filed, the company has published two blog posts that seem to reference the case. The first pointed to the “recent heartbreaking cases of people using ChatGPT in the midst of acute crises.” The company declared that it had built guardrails into ChatGPT to respond to users in psychological distress, but that these safeguards “may degrade” during lengthier conversations. It also said that parental controls were in development, and would include the option for teenagers to upload emergency contacts to the chatbot should a conversation veer into dangerous territory. In the second post, OpenAI touted the “council” of mental health and A.I. experts it had convened to improve the safety of its product.

Edelson PC, the law firm bringing the suit, found the blog posts telling. “That is not the response that you issue to the world if you think this is an isolated incident, saying that, Yes, we understand that there are issues with the safeguards,” Eli Wade-Scott, one of the lead attorneys on the case, told me. In the week since filing the lawsuit, he added, Edelson has been inundated with similar inquiries from people saying they’ve experienced harm at the “hands” of a chatbot. Scott said that the volume of outreach has “really been shocking.”

He said he expects OpenAI to argue that chatbot speech is protected by the First Amendment and raise state law–related issues with the claims. Scott was also adamant that “the family is doing this because they don’t want to see this happen to any other family ever again. They’re looking for changes to the product.” He pointed me to a foundation the family started in their son’s name, which “seeks to raise awareness around the kind of risks to teens from ChatGPT.”

Aaron Davis, a personal injury attorney at Davis Goldman, told me there are a few key questions surrounding the case, beyond the somewhat straightforward theory of liability that holds a company responsible if a product proximately causes injury. The first contends with whether ChatGPT can legally be considered a product at all, even though OpenAI has used that term to describe it. The question, he said, is complicated by the fact that the system is free—although, in this case, Adam Raine was subscribed to ChatGPT Plus, which costs $20 per month. Paying in exchange for some defined service, according to Davis, would better circumscribe the issue at hand.

But the main question, Davis said, had to do with “reliance” on the technology: “Would a reasonable person be expected to rely upon that information in making important life decisions? And does OpenAI understand that people are doing that, and disregard that function that they are enabling?” This is where the case might get nebulous, since the notion of “reliance” as it applies to a chatbot like ChatGPT is both subjective and has yet to be legally established. But as adoption increases, “I think that the liability will continue to accrue,” he added.

Finally, Davis noted that the powerful relationships forming between people and chatbots have introduced a new legal dimension: “What if I was getting online therapy through a psychiatrist, and a psychiatrist gave me advice about how to handle my life, and suggested that I should kill myself? Certainly you would have established privity in a relationship of reliance there.” This strain of argument raises an entirely different set of questions: Is OpenAI dispensing medical advice without a license? Or is OpenAI simply providing medical information, similar to WebMD?

While OpenAI’s terms state that its output should not be relied on as a substitute for “professional advice,” they do not specifically mention medical advice or therapy. In the end, Davis expects this litigation will result in the introduction of clear terms and conditions “that precludes A.I. being responsible for certain things—essentially some sort of waiver relationship that creates that barrier of insulation.”

On August 25, the day before the Raine family lawsuit was filed, the National Association of Attorneys General published an open letter addressed to OpenAI, Anthropic, Google, Microsoft, Character AI, Meta, and xAI, among others. In powerful language, it warned that companies ignoring the risk to children will “answer for it.” According to the letter, “As the entities benefitting from children’s engagement with your products, you have a legal obligation to them as consumers. And conduct that would be unlawful—or even criminal—if done by humans is not excusable simply because it is done by a machine.”

A few days later, California A.G. Rob Bonta and Delaware A.G. Kathy Jennings wrote to OpenAI’s board to express their “serious concerns” with OpenAI’s safety procedures, calling the recent deaths “unacceptable.” The Federal Trade Commission is also reportedly preparing to question OpenAI and other developers about the impact their chatbots have on kids.

Meanwhile, the Center for Humane Technology published a blog arguing that what happened to Adam was a direct result of purposeful design choices made by OpenAI. The chatbot was designed, the group said, to continue surfacing new methods of engagement, rather than answering Adam’s homework questions and simply ending the conversation there. Combined with the anthropomorphic, human-like tendencies baked into ChatGPT, the center argued that this resulted in “a parasitic relationship.” For its part, OpenAI has stated that the company’s goal “isn’t to hold people’s attention.” And yet, its own research has linked emotional dependency and loneliness with greater use of its chatbot.

As the case gets underway, Edelson is calling for OpenAI to either affirm that ChatGPT is safe or “pull it from the market.” That seems unfathomable, but perhaps these lawsuits will ensure that these tragedies become far less common.

That’s all for today. I’ll see you on Thursday.

Ian

Need help? Review our FAQ page or contact us for assistance. For brand partnerships, email ads@puck.news.

You received this email because you signed up to receive emails from Puck, or as part of your Puck account associated with {{customer.email}}. To stop receiving this newsletter and/or manage all your email preferences, click here.

Puck is published by Heat Media LLC. 107 Greenwich St, New York, NY 10006