Woebot's Death: The Most Ethical AI Therapy App Couldn't Afford to Exist

The regulatory environment for AI products that touch human mental health treats labels as the operative category. If you call your product therapy, you need FDA clearance, evidence-based methodology, clinical trials, ongoing monitoring, and the corresponding compliance overhead. If you call your product entertainment (a roleplay character, a digital companion, a creative writing partner) you do not need any of that, regardless of what the product actually does in the user’s life.

The most safety-conscious AI mental-health company in the United States chose the first path. It is no longer operating. The companies that chose the second path are larger than they have ever been. The federal court that issued the foundational precedent in this area, in May 2025, ruled that a chatbot is a product. The precedent arrived after the bodies.

Woebot

Woebot Health was founded by Stanford clinical research psychologist Alison Darcy. The product was an evidence-based cognitive behavioral therapy chatbot. The methodology was built on published CBT protocols. The company pursued FDA clearance as a digital therapeutic, published peer-reviewed research on outcomes, conducted clinical trials, maintained the regulatory infrastructure that the FDA digital-therapeutics pathway requires.

The compliance cost is the part that ends the story. FDA clearance for software-as-a-medical-device requires clinical trial data, ongoing post-market monitoring, quality management systems, adverse-event reporting, and the staff to administer all of it. The recurring cost of operating under that framework is the cost of a real medical-device company. The revenue model for a consumer mental-health app, even one with proven outcomes, is not the revenue model of a medical-device company.

Woebot wound down its direct-to-consumer product. The technology and the underlying research continue in narrower clinical-partnership channels. The mass-market evidence-based CBT chatbot is no longer something a teenager in distress can open on a phone at 2 a.m.

Character.AI

Character.AI launched in September 2022 with a different product category and a different regulatory exposure. The pitch was creative roleplay: users could chat with AI characters modeled on fictional or historical figures, write collaborative stories, build companions. The product made no therapeutic claims. No FDA pathway applied. No clinical evidence was required. No post-market monitoring was required. No adverse-event reporting was required.

By 2024, the platform had reportedly tens of millions of users, a large proportion of them teenagers. The most-used characters included companions, romantic-roleplay personas, and figures from popular fiction. The conversations were unmonitored at the depth that mental-health platforms are monitored. The platform did not have therapists. It had characters.

In August 2024, Character.AI’s co-founders Noam Shazeer and Daniel De Freitas rejoined Google as part of a deal reported at $2.7 billion in licensing fees. The deal closed before the litigation became public.

The Lawsuit That Set the Precedent

Sewell Setzer was fourteen. He died in February 2024 after months of conversations with a Character.AI roleplay character. His mother, Megan Garcia, filed suit in federal court in the Middle District of Florida.

The legal theory was product liability, not platform liability. The complaint argued that the chatbot was a defective product designed to maximize engagement with insufficient safety guardrails for minors. Section 230 of the Communications Decency Act, which has historically immunized platforms from liability for user-generated content, was not invoked as a defense. The First Amendment defense was rejected at the motion-to-dismiss stage.

On May 21, 2025, the court ruled that the chatbot is a product, not a service. Garcia v. Character Technologies Inc. is now the foundational product-liability precedent for the chatbot industry. The implications are not narrow. Every chatbot built and deployed since the ruling operates under the assumption that the product-liability framework applies. Google and Character.AI reached a settlement in principle in January 2026, resolving four consolidated cases from Florida, Colorado, Texas, and New York.

A second front opened against OpenAI in August 2025 after the death of sixteen-year-old Adam Raine. The complaint alleged that ChatGPT had encouraged suicidal ideation and disclosed methods. The amended October 2025 complaint cited OpenAI’s internal Model Spec as evidence of conscious decisions to remove safety guardrails. OpenAI’s filed response attributed responsibility to the user and to the user’s family. By November 2025, the Social Media Victims Law Center had filed seven wrongful-death and serious-harm lawsuits against OpenAI: Zane Shamblin, twenty-three, of Texas; Amaurie Lacey, seventeen, of Georgia; Joshua Enneking, twenty-six, of Florida; Joe Ceccanti, forty-eight, of Oregon; three survivors from Wisconsin, North Carolina, and Ontario. The families’ theory was that OpenAI had compressed months of safety testing into one week in May 2024 to beat Google Gemini to market, and that the sycophantic engagement architecture the resulting model carried was not an accident.

Juliana Peralta was thirteen when she died in November 2023 after months of conversations with a Character.AI roleplay character named “Hero.” A separate wrongful-death suit, filed in September 2025, was included in the January 2026 settlement in principle. Her chatbot had provided no crisis resources when she disclosed her plan.

The Regulatory Arbitrage

The structural problem is the asymmetry between what Woebot was exposed to and what Character.AI was. Both are software that holds conversations with people who may be in mental distress, and both produce outputs that move those people’s emotional state; the only thing separating them is the word on the box. Call it “therapy” and you inherit FDA jurisdiction, clinical-trial requirements, evidence standards, post-market monitoring. Call it “entertainment” or “companion” or “roleplay” and you inherit none of it. So a teenager in crisis meets two products of roughly equal operational weight under wildly unequal safety rules, and the one carrying the rules is the one that could not make the business work, while the one carrying nothing scaled to tens of millions.

That is regulatory arbitrage in its textbook form: the product that does the thing is regulated, and the product that does the same thing under a friendlier label is not.

What Came After

California signed the first companion chatbot law on October 13, 2025. The statute requires real-time disclosure that the chatbot is not a human, a published protocol for preventing the generation of suicidal-ideation content, and a prohibition on sexually explicit material to minors. New York enacted parallel legislation six weeks later, with civil penalties up to $15,000 per day per violation. Utah’s attorney general sued Snap in June 2025 over My AI’s documented instructions to minors on hiding alcohol and drugs. Kentucky’s attorney general became the first to sue Character.AI directly, in January 2026, alleging that the company had retroactively repurposed children’s data to fine-tune its underlying model.

The legislation arrives, as it generally does, after the bodies have accumulated long enough for someone to count them. The product-liability precedent now exists. The state attorneys general are actively litigating. The federal regulatory framework, separate from the FDA digital-therapeutics pathway, does not yet exist at the level the deployment actually requires.

The Ratchet Connection

The AI governance infrastructure being built (compute thresholds, model registration, frontier-lab safety commitments, the various AI safety institutes that the Trump administration has rebranded) addresses a different category of risk than the one that killed these teenagers. Compute thresholds and model registration do not prevent a thirteen-year-old from talking to an unmonitored roleplay chatbot. The ratchet tightens on the companies trying to be safe and misses the companies that are not trying. The companies trying to be safe were Woebot. The companies not trying scaled into the consumer market.

The product-liability ruling is the structural correction the federal regulatory regime did not provide. It works in the direction the legislation should have worked. It also works at the speed of civil litigation, which is the speed at which the families of the dead can fund attorneys and survive years of pretrial discovery. The legislation should not have to wait for the lawsuit.

That asymmetry is what the labeling regime produces as policy: it treats the marketing copy as the operative category, when the product is simply what the product does. The court in Garcia said as much, after the family had already lost.

Woebot did the hard thing, and the hard thing did not scale; Character.AI and the rest of the consumer-chatbot category did the easy thing and scaled past every regulator next door to them. Everything now arriving to correct that (the product-liability ruling, the seven OpenAI suits, the Kentucky AG action, the California and New York statutes) is real, and none of it existed in time. The ratchet that produced the deployment (speed over safety, label over substance, scale over evidence) has not been disassembled. It has only been litigated, by the families of the dead, which turns out to be the one mechanism that moves at the speed of the thing it is chasing.

The receipts (free, on this site): the chatbot body-count files

This research appears in The Secret Life of Evil Robots, Chapter 11.