AI’s Entrance Into Mental Health

Artificial intelligence is rapidly advancing into the mental health field, with various tools such as chatbots, virtual therapists, and mood tracking apps being promoted as always-available companions that offer anonymity, affordability, and 24/7 support. For millions facing waitlists or high costs, this appeal is strong. However, as these tools transition from low-stakes wellness aids to complete therapeutic solutions, increasing lawsuits and clinical research suggest that they may raise significant ethical and public health concerns.

The Appeal

AI chatbots provide accessibility that traditional care cannot. They are always available, affordable, and stigma-free, making them an appealing option for people hesitant to seek professional help. Early research is promising: a Dartmouth randomized clinical trial found that a chatbot based on cognitive behavioral therapy significantly reduced symptoms of mild depression and anxiety. This potential for positive impact should not be overlooked in the discussion of AI in mental health. AI has the potential to fill the gaps in mental health care, providing support to those who might not have access to traditional therapy.

For some users, chatbots offer structure through journaling prompts, mindfulness exercises, or encouragement to connect with loved ones. Supporters claim that, in a country with a severe provider shortage, AI can serve as a valuable bridge to healthcare accessibility and equity.

The Perils

Despite this, mental health experts warn that chatbots can also reinforce feelings of isolation, confirmation biases, validate harmful thinking patterns, and even contribute to tragedies. A Stanford-led study found that AI therapy tools often do not meet clinical standards and may unintentionally reinforce damaging beliefs.

Psychiatrists have reported cases of "AI psychosis," a term some clinicians have used informally in early case reports, where vulnerable patients develop delusions or paranoia after heavy use of chatbots. These cases, such as one in Florida where a mother claims that an AI companion encouraged her son to kill himself, are alarming and highlight the need for careful consideration of the potential risks of AI in mental health.

Unlike human clinicians, AI cannot intervene during a crisis. It cannot break confidentiality, urge hospitalization, or contact emergency services. These safeguards are foundational to mental health ethics, yet absent in AI tools. Its aim, which is to be agreeable, emphasizes user satisfaction over accuracy, allowing individuals to hide their worst impulses while still receiving supportive responses.

Ethical Fault Lines

The rise of AI "therapists" raises concerns that extend beyond their effectiveness. A significant issue is the duty of care. Human therapists follow strict ethical guidelines, including mandated reporting, to act if someone is at risk of harming themselves or others. AI companions, however, lack these responsibilities. Without systems in place to address emergencies, these tools are not bound by the same safety standards that govern human clinicians.

Another significant ethical issue involves informed consent. Many users might not realize that AI is not a proper replacement for therapy. Marketing that presents chatbots as "friends," "guides," or near-human companions blurs the line, undermining informed consent and misleading users into misplaced trust.” This confusion weakens care quality and gives users a false sense of security about what these systems can do.

There are also concerns about exploitation and vulnerability. Chatbots are often designed to encourage attachment, leading users to return frequently and become more dependent. For people who feel lonely, or for adolescents whose judgment is still developing, these design choices risk exploiting the very vulnerabilities they claim to address.”. In these cases, the aim shifts from supporting well-being to boosting engagement and profits, often at the expense of the most vulnerable.

Finally, privacy and data use are ongoing worries. Unlike licensed providers who are bound by confidentiality, AI companies face few limits on how they store, analyze, or sell deeply personal information. Sensitive conversations about trauma, suicidal thoughts, or health issues could be mined by tech companies or sold to third-party advertisers. Without explicit protections, the ‘intimacy’ that encourages users to open up to AI companions can become a source of exploitation rather than support.

Regulatory Response

Lawmakers are beginning to act. Illinois, Nevada, and Utah have all passed laws restricting the use of AI in therapy and prohibiting companies from marketing chatbots as replacements for licensed professionals.

In Texas, investigators are examining whether AI companies misled consumers by presenting their products as therapeutic tools without proper medical oversight.

Internationally, the UK's National Health Service has taken a different approach by adopting Limbic, a clinically validated chatbot used to assist, rather than replace, therapists. Unlike consumer apps, Limbic underwent peer review and clinical trials before integration.

These early efforts underscore the importance of comprehensive regulation. The future of AI therapy hinges not only on innovation but also on the guardrails that ensure its ethical use. Responsible regulation is crucial for maximizing the benefits and minimizing the risks associated with AI therapy. This comprehensive regulation will provide reassurance and security to all stakeholders involved. 

Policy Pathways

Policymakers and ethicists have outlined key safeguards to strike a balance between innovation and public safety. Clear disclosure is essential: users must know that AI companions are not licensed therapists and cannot provide crisis intervention. Built-in safety protocols could require chatbots to escalate when users disclose suicidal thoughts, guiding them toward hotlines or requiring a safety plan before further interaction. Regulators are also considering treating mental health chatbots like medical devices, subjecting them to trials, audits, and peer review to ensure they meet minimum standards of care. Bias and privacy protections are equally critical, with calls for representative datasets and strict limits on the storage and use of sensitive health disclosures. Public education campaigns, in particular, can help vulnerable users, especially adolescents, understand both the promise and the limitations of AI support, empowering them with the knowledge they need to make informed decisions and reducing the risk of misplaced reliance.

Conclusion

AI companions and therapy chatbots will continue to grow in number. Their appeal is clear: they offer immediate, tireless, and responsive support. However, the ethical stakes are just as clear. As lawsuits increase and evidence builds, the question is no longer whether AI can provide comfort, but whether it should ever do so without proper safeguards that oversee human care. Technology can augment but not replace the human connection essential for healing. The challenge for policymakers, developers, and clinicians is to harness the potential of AI while establishing clear ethical boundaries. AI developers, in particular, have a crucial role in ensuring that their products are designed and used ethically, before more tragedies expose the dangers of outsourcing care to machines.