When the Therapist Becomes Code: What We Must Demand Before We Trust Digital Souls

## Introduction Technology is changing how we seek help for mental health issues. Chatbots and systems using generative artificial intelligence are increasingly claiming roles as support, advisor, or even "therapist." This raises both excitement and deep concern. In this article I take a clear position: we should not dismiss the potential of digital tools for mental health, but we must demand evidence, regulation, and ethical frameworks before we let them take over vulnerable encounters between human beings. This is an opinion piece, but it draws on concrete findings and recent reports about treatment effects in mental health and neuropsychiatric conditions. As an early example of why rigorous research matters, new research on ADHD treatment shows how analyses can change our understanding of what actually works in clinical practice. According to VG (published August 3, 2025), "New analysis shows what may give the best effect on ADHD symptoms" and that this research "will increase our understanding." This kind of insight should be a prerequisite for the development and deployment of AI therapeutic solutions — not the opposite. This article explores what research says, what generative AI can realistically offer, the concrete risks and ethical challenges that follow, and how we as a society must regulate and integrate the technology responsibly. The goal is to present a balanced but clear stance: digital tools can be useful, but they must not get a free pass because they are new or convenient. ## What treatment research teaches us — and why it matters for digital tools Research on mental disorders and treatments is often complex; small differences in design, population, or measurement methods can produce very different results. As VG's coverage of new ADHD research notes — stating that "new analysis shows what may give the best effect on ADHD symptoms" — this illustrates how important methodological rigor is. When researchers say a new analysis "will increase our understanding," that is not just academic language — it signals that treatment effects must be carefully reproducible and that conclusions must be based on robust data. Translating this mindset to the field of generative AI and mental health makes the implications obvious: we cannot roll out "AI therapists" at scale without randomized controlled trials, longitudinal follow-up data, and clear definitions of what "effect" means. In ADHD research, analyses focused on which interventions actually reduce symptoms or improve functioning; the same must apply to chatbot psychologists and other digital aids. Is symptom reduction measurable? What side effects occur? How do effects vary by age, gender, socioeconomic status, or comorbidity? The VG article reminds us that the key to progress is knowledge — knowledge rooted in systematic research, not anecdotes. Therefore developers, health authorities, and researchers must collaborate to define evidence standards for "artificial intelligence therapy" and other digital solutions. For patients and clinicians it should be as easy to find and trust evidence for an AI-based intervention as it is for medication or cognitive therapy. ## What generative AI can offer mental health — a realistic promise, not a cure-all Generative AI can process large amounts of text and tailor responses in real time, offering several potential uses in mental health: low-threshold support outside appointment hours, structured follow-up between consultations, help tracking and reflecting on mood and behavior, and support where clinic capacity is lacking. An "AI therapist" can also assist with psychotherapeutic home follow-up, medication or behavior-change reminders, and provide first-line support in acute but non-life-threatening situations. However, we must be realistic about what these systems can do. They can supplement but not replace human judgment in complex or acute cases. Their strength lies in accessibility, scalability, and the ability to provide consistent follow-up. For example, a chatbot psychologist may be useful for teaching users techniques from cognitive behavioral therapy through structured exercises and reminders. It is nonetheless crucial that such tools are integrated into a treatment plan that includes human oversight when needed. At the same time, generative AI can contribute to research. Machine learning can identify patterns in large datasets that humans miss, similar to how new analyses in the ADHD field can shed fresh light on effective interventions. But here is an important caveat: analyses must be interpreted correctly, and models must be validated on new, independent datasets before we conclude that an observation is clinically relevant. Otherwise we risk building treatment on random correlations. ## Risks and ethical dilemmas: Where technology fails people The spread of "AI therapist" and chatbots in mental health raises a number of concrete risks. First and foremost is privacy and data access. Mental health information is among the most sensitive data we have. If a chatbot psychologist collects, stores, or shares data without adequate consent or security, the consequences for the user can be severe. The VG article on ADHD research indirectly warns us against taking data security lightly in health contexts: insight must be based on quality and responsibility, not merely availability. Another problem is model errors and "hallucinations." Generative AI can invent plausible-sounding answers, which in a therapeutic context can be dangerous. Imagine a case where a chatbot gives incorrect advice about medications, downplays risks, or minimizes suicidal thoughts. Even if such systems are designed to escalate in cases of acute risk, those mechanisms may not work flawlessly in all situations or language variants. Bias and unequal access present further ethical challenges. AI models are trained on data that often reflect systematic societal biases. This can lead to minorities, people from diverse cultural backgrounds, or those with rare comorbidities receiving poorer or incorrect help. Commercialization also introduces the risk of profit-driven solutions that prioritize user engagement and monetization over patient safety. Finally, there is an existential concern: dependency and dehumanization. As users increasingly prefer automated, always-available services, the relationship with human therapists can weaken. Therapy is not just techniques; it is the relationship. We must ask what we are willing to sacrifice for convenience. ## How to ensure safe and effective implementation: the checklist we should insist on To balance potential and risk requires a set of clear demands and measures. First and foremost: evidence-based deployment. Any tool marketed as an "AI therapist" or "chatbot psychologist" must undergo clinical studies that document effectiveness, safety, and side effects. Such research should meet the same standards as other medical interventions. As VG points out in reflecting on ADHD analyses, robust research increases our understanding — this must also be the goal for generative AI in mental health. Second: transparency and accountability. Providers must make clear what data are collected, how they are used, and who has access. Model limitations should be explicitly communicated to users and professionals. There must also be lines of responsibility: who is clinically responsible when a chatbot gives advice? Who monitors and updates the model? Third: emergency procedures and integration into healthcare. Systems must include built-in risk-assessment protocols that lead to rapid escalation to a human professional in cases of self-harm risk, severe deterioration, or complex symptom pictures. They should not operate in isolation; the ideal is support tools that work in concert with mental health professionals. Fourth: regulation and quality control. Authorities must develop rules that address safety, data protection, and documentation requirements. This includes model certification, transparency requirements, and oversight mechanisms. Industry standards and independent evaluation bodies can play a role, as they do in other medical fields. Finally: ethical design and inclusion. Developers must ensure that the data used to train systems are representative and that models are tested across diverse demographic groups. User involvement in the design phase — especially from marginalized groups — is essential to avoid biases that could reinforce inequality. ## Practical recommendations for clinicians, policymakers, and users For clinicians: view generative AI as a potential supplement, not a replacement. Demand documentation and safety from providers, and participate in multidisciplinary assessments before implementing tools in practice. Train staff on how systems should be used, when to escalate to human review, and how to recognize incorrect or harmful recommendations. For policymakers: prioritize frameworks that require clinical validation and ensure privacy. Fund independent research so that commercial interests do not alone define what works. Trigger warnings and regulate systems that offer clinical recommendations without documented effect. For users: be critical. Ask what documentation the provider can show. Read privacy policies and consider the consequences of sharing sensitive information. Most importantly: if you experience severe symptoms or suicidal thoughts, contact healthcare professionals or emergency services — AI is not a replacement in such situations. ## Conclusion: A pragmatic and demanding path forward Technology has real potential to extend the reach of mental health care and provide immediate, low-threshold support to many. Still, we must not let technological enthusiasm overshadow fundamental requirements for safety, evidence, and ethics. According to VG (August 3, 2025), new analysis "will increase our understanding" of what works in treatment — this is a lesson we must carry forward. Developing and rolling out "AI therapist" and "chatbot psychologist" solutions requires the same research-based rigor and caution as any other medical innovation. My clear position is that we should support responsible innovation, but also be prepared to set limits. We must insist on clinical documentation, data protection, transparency, and regulation. We must avoid a situation where marketed solutions without documented effect fill a gap that should be addressed by public healthcare. If we succeed in combining technological benefits with rigorous scientific testing and ethical reflection, generative AI can become a valuable supplement in mental health. But until then: demands for knowledge must come first. ## Sources - VG: "Research on ADHD treatment: – Will increase our understanding." Published August 3, 2025. According to VG: "New analysis shows what may give the best effect on ADHD symptoms" (https://www.vg.no/helse/i/EyE9Ro/forskning-om-adhd-behandling-vil-oeke-vaar-forstaaelse)