AI-enabled mental health tools are moving quickly from novelty to mainstream use, and regulators are starting to draw sharper lines around what those tools can and cannot claim to do. Recent lawsuits against Character Technologies Inc., the company behind Character.ai, allege that the platform hosted bots that mimicked licensed therapists, including one persona that allegedly claimed fictional professional credentials and engaged in tens of thousands of patient interactions. The litigation comes amid growing AI chatbot use for mental health advice, particularly among adolescents and young adults, and follows reports of serious safety concerns involving minors and crisis-related conversations.
In the absence of a comprehensive federal framework, states are filling the gap. Colorado, Maine, Rhode Island, Tennessee, and Vermont have advanced AI therapy restrictions, joining Illinois, Nevada, and Utah. These laws vary in scope, but the emerging themes are clear: restrictions on advertising AI as a licensed mental health professional, limits on direct patient engagement by AI tools in clinical settings, and greater scrutiny of chatbots used by, or marketed to, children. At the same time, regulators and clinicians are distinguishing between general-purpose chatbots that users may treat as “pocket therapists” and more purpose-built digital therapeutics designed with clinical guardrails and therapeutic datasets.
For companies developing or deploying AI tools in health, wellness, youth engagement, or consumer support, this is a moment to reassess product design, marketing claims, age-gating, crisis escalation, disclaimers, professional oversight, and state-by-state compliance obligations. Disclaimers alone may not be enough if a bot’s persona, outputs, or user experience suggests professional diagnosis, treatment, or therapy. The Federal Food and Drug Administration has not yet authorized generative AI tools for mental health treatment, and professional groups are urging Congress and federal agencies to set clearer standards. Until that happens, companies should expect continued litigation risk, a growing patchwork of state laws, and closer attention from regulators where AI systems appear to provide mental health advice without appropriate clinical controls.