AI & Mental Health: Mind Launches Inquiry into Google’s Harmful AI Advice

A year-long inquiry into the impact of artificial intelligence on mental health has been launched by the UK mental health charity Mind, following a Guardian investigation revealing potentially “very dangerous” advice generated by Google’s AI Overviews.

The commission, described as the first of its kind globally, will bring together leading doctors, mental health professionals, individuals with lived experience, healthcare providers, policymakers, and technology companies. The goal is to shape a safer digital mental health ecosystem through stronger regulation, standards, and safeguards, as AI increasingly influences the lives of those affected by mental health issues.

The impetus for this inquiry stems from concerns about the accuracy and potential harm caused by AI-generated summaries appearing above traditional search results on Google – a platform accessed by an estimated 2 billion people monthly. Rosie Weatherley, information content manager at Mind, the largest mental health charity in England and Wales, articulated the core problem: the way AI Overviews present complex and nuanced mental health information as definitive answers.

“Over three decades, Google designed and delivered a search engine where credible and accessible health content could rise to the top of the results,” Weatherley explained. “Searching online for information wasn’t perfect, but it usually worked well. Users had a good chance of clicking through to a credible health website that answered their query.”

However, Weatherley argues that AI Overviews have fundamentally altered this dynamic. “AI Overviews replaced that richness with a clinical-sounding summary that gives an illusion of definitiveness. It’s a very seductive swap, but not a responsible one. And this often ends the information-seeking journey prematurely. The user has a half answer, at best.”

Mind conducted an internal test, tasking its team of mental health information experts with using search queries commonly employed by individuals experiencing mental health challenges. The results were deeply concerning. Within minutes, the AI provided demonstrably false and harmful information.

“Within two minutes, Google had served AI Overviews that assured me starvation was healthy,” Weatherley recounted. “It told a colleague mental health problems are caused by chemical imbalances in the brain. Another was told that her imagined stalker was real, and a fourth that 60% of benefit claims for mental health conditions are malingering. It should go without saying that none of the above are true.”

The issue, Weatherley emphasizes, isn’t simply the presence of inaccurate information, but the way AI Overviews present it. “In each of these examples we are seeing how AI Overviews are flattening information about highly sensitive and nuanced areas into neat answers. And when you take out important context and nuance and present it in the way AI Overviews do, almost anything can seem plausible.”

This simplification is particularly dangerous for individuals already experiencing distress. The AI’s confident and authoritative tone lends a false sense of credibility to potentially harmful claims. Weatherley noted that the process is especially harmful for people who are likely to be in some level of distress.

Weatherley also raised concerns about Google’s reactive approach to addressing these issues. “A multi-billion-dollar company like Google that profits from AI Overviews should have more resources dedicated to providing accurate information. The extent of their concern seems limited to reactively retraining or removing AI Overviews when individuals, organisations or indeed journalists flag new insights. This ‘whack-a-mole’ style of problem-solving feels unserious and not scaled to the size and resource of the company profiting from them.”

The charity also highlighted the potential for AI to undermine existing safeguards designed to limit access to harmful content, such as information about suicide methods. While search engines have made efforts to reduce the visibility of such material, Weatherley warned that AI Overviews could circumvent these protections, presenting dangerous information in a seemingly neutral and authoritative manner. “Search engines have evolved to make access to the most harmful search results, like suicide methods, less immediately available. But if you search as an unwell person might search, the risk remains that you will be served harmful inaccuracies and half-truths, presented in calm and confident copy as uncontroversial neutral facts with the stamp of approval from the world’s biggest search engine.”

Weatherley pointed to the inconsistent and often contradictory nature of crisis information provided by AI Overviews. “In a search for crisis information, the AI Overview haphazardly collaged various contradictory signposts in long lists.”

While acknowledging the potential benefits of AI in healthcare, Mind’s inquiry underscores the urgent need for responsible development and deployment. “Perhaps AI has enormous potential to improve lives, but right now, the risks are really worrying,” Weatherley concluded. “Google will only protect you from the potential faults of AI Overviews when it thinks you’re in acute distress. People need and deserve access to constructive, empathetic, careful and nuanced information at all times.”