II · D13 · 13.1
Mental-health chatbot safety
Crisis handling failure modes.
Major · 4Widely observedNowRisk 16
Severity
4/5
Likelihood
4/5
In the hierarchy
- Part II: Societal & Human Concerns
- D13. Health, Safety & Care
- 13.1 Clinical AI
Virtues and principles to explore
Held as inquiry, not as a verdict.
Service
Hope
Who is served and who benefits are not always the same.
Dignity
Center principle
Every person is a protagonist, not a data point.
Listening
Trust
Listen to understand, rather than listen to speak.
Starter questions
Written to probe curiosity and learning, not accusation.
- 01How would we know we were getting this wrong? What would we observe, without blaming a person?
- 02What would trustworthiness look like here: what we do, what we say, and the belief others form?
- 03What would it look like if we held “Mental-health chatbot safety” with proportion rather than alarm?
- 04If we named this complication in advance, what small, reversible step would let us learn?
Also on this branch
Frameworks and sources
- WHO guidance on ethics and governance of AI for healthframework
- EU AI Act — Annex III high-riskregulation
See also exemplar policies.
Writing and incidents
Open indexes first. Then, if you wish, ask Grok to search the live web for this concern — one request, cached for the rest of this session.
Care, clinics, and the people who depend on systems they did not choose.