Active Mental Health Computing & AI

SAFER-MH: Improving Safety Evaluation and Regulatory Capacity for AI-Enabled Mental Health Care

In plain English

AI plain-English summary

South Africa’s medicines regulator is getting a new framework to evaluate AI-powered mental health apps and devices, alongside tools to detect the specific harms these technologies can cause. Mental health apps and chatbots are proliferating, but regulators have no established way to assess their safety or effectiveness. The same AI that can triage depression symptoms or deliver therapy could also give dangerous advice, reinforce biases, or mishandle sensitive user data. Without clear rules, developers either avoid the market or launch products with unknown risks. This project tackles that gap head-on. The researchers will build a regulatory pathway with SAHPRA, define a taxonomy of AI-specific harms in mental health, and create a standard for logging user engagement data so outcomes can be tracked. The work focuses on low-resource settings, where AI could expand access to care but regulatory capacity is thinnest. If successful, the framework could reduce the cost and complexity of compliance for developers while giving regulators a repeatable method to evaluate safety. Because the field is so new, the outputs—harm taxonomies, data standards, regulatory templates—will likely be adopted globally, shaping how AI is governed in mental health from the ground up.

View original technical description
In this proposal, we aim to address three key challenges to ensure that AI has safe and equitable impact in mental health (MH): we and our partners will collaborate with the South African Health Products Regulatory Agency (SAHPRA) to create a regulatory pathway for AI-enabled (MH) products; we will define a taxonomy of potential harms caused by AI-enabled MH products and develop new ways of detecting those harms; and we will establish a standard for how information about user engagement is recorded so that we can draw better links between products and real-world outcomes. Taken together, these three strands of work will help developers monitor and maintain user safety, regulators understand how to evaluate the safety of AI applications, and patients access quality care. More succinctly, our work will reduce the friction, complexity, and cost associated with regulatory compliance, making it easier for developers to innovate safely. Our focus will be to deliver these solutions in low-resource settings, but since the application of AI to MH and the field of regulatory science for AI-as-a-medical-device is so nascent, many of our outputs will have global relevance for advancing these fields.

View the original record at the funder ↗

Researchers

Alastair Denniston (EPMC Awardee)Alastair van Heerden (EPMC Awardee)Bilal Mateen (EPMC Awardee)Christelna Reynecke (EPMC Awardee)Dimakatso Mathibe (EPMC Awardee)Russell Pearson (EPMC Awardee)Sarah Morris (EPMC Awardee)Tim Althoff (EPMC Awardee)Xanthe Hunt (EPMC Awardee)

Related Research

Grants with similar aims, by meaning.

Effective Regulation and Evaluation of Digital Mental Health Technology
Governing AI safety in healthcare: developing a learning architecture for regulation, investigation and improvement
Ensuring the benefits of AI in healthcare for all: Designing a Sustainable Platform for Public and Professional Stakeholder Engagement
Collaboratively developing an AI-driven patient safety formulation tool to enhance healthcare services self-harm and suicide prevention
Extension - Effective regulation and evaluation of digital mental health technologies

Original classification

Discretionary Award

Plain English summaries and category classifications on this site are generated by AI and may not perfectly reflect the original research.