Google AI Health Advice: Hidden Warnings Put Users at Risk

Google’s recently implemented AI Overviews, designed to provide quick answers to search queries, are raising concerns among AI experts and patient advocates regarding the presentation of medical information. While Google states it prompts users to seek professional medical advice when appropriate, a recent investigation reveals that crucial disclaimers about the potential inaccuracies of AI-generated health information are often hidden from initial view, potentially putting users at risk.

The core of the issue lies in the placement of these disclaimers. Instead of appearing alongside the initial AI-generated summary – the information users see directly after posing a health-related question – they are relegated to a “Show more” button. Even then, the disclaimer, stating “For medical advice or a diagnosis, consult a professional. AI responses may include mistakes,” is presented in a smaller, lighter font at the very end of the expanded overview. This delayed and subtle presentation raises questions about whether users are adequately informed about the limitations of the information they are receiving.

Google maintains that its AI Overviews “encourage people to seek professional medical advice” and often include such recommendations within the summary itself. However, critics argue that this is insufficient. The immediacy and authoritative tone of the AI Overview, appearing at the top of search results, can create a false sense of security, discouraging users from seeking further information or consulting a healthcare professional.

The Risks of Immediate Reassurance

The potential for harm stems from several factors. Even sophisticated AI models are prone to “hallucinations” – generating incorrect or misleading information – and can prioritize user satisfaction over accuracy. In the context of health, this can have serious consequences. Users may not articulate their symptoms accurately or provide sufficient context when querying the AI, leading to inaccurate or incomplete responses.

“The absence of disclaimers when users are initially served medical information creates several critical dangers,” explains Pat Pataranutaporn, an assistant professor and researcher at the Massachusetts Institute of Technology specializing in AI and human-computer interaction. “Disclaimers serve as a crucial intervention point. They disrupt this automatic trust and prompt users to engage more critically with the information they receive.”

This disruption of “automatic trust” is particularly important because the AI Overviews are designed for speed and convenience. Users seeking quick answers may be less likely to scrutinize the information presented or to click through to more detailed sources, where a disclaimer might be found. As Sonali Sharma, a researcher at Stanford University’s center for AI in medicine and imaging, points out, the AI Overview often presents a “complete answer” immediately, discouraging further investigation.

A Design Problem, Not Just an AI Limitation

The issue isn’t solely about the inherent limitations of AI technology. Gina Neff, a professor of responsible AI at Queen Mary University of London, argues that the problem is “by design.” “AI Overviews are designed for speed, not accuracy and that leads to mistakes in health information, which can be dangerous.” This prioritization of speed over accuracy raises ethical concerns about the responsibility of tech companies when providing health-related information.

This isn’t a new concern. Earlier this year, investigations revealed instances of false and misleading health information appearing in Google AI Overviews, prompting Google to temporarily remove the feature for some medical searches. However, the underlying issue of disclaimer placement remains unresolved.

The Importance of Prominent Disclaimers

Patient advocates emphasize the need for a more prominent and visible disclaimer. Tom Bishop, head of patient information at Anthony Nolan, a blood cancer charity, stresses the importance of prompting users to pause and consider the source of the information. “That disclaimer needs to be much more prominent, just to make people step back and think… ‘Is this something I need to check with my medical team rather than acting upon it? Can I take this at face value or do I really need to look into it in more detail and see how this information relates to my own specific medical situation?’”

Bishop advocates for a disclaimer that is “right at the top,” presented in the same font size as the rest of the information. This would ensure that users are immediately aware of the potential limitations of the AI-generated advice.

The Challenge of Partially Correct Information

The danger is amplified by the fact that AI Overviews often contain a mix of accurate and inaccurate information. Without a strong understanding of the subject matter, it can be difficult for users to discern what is reliable and what is not. This is particularly concerning for individuals seeking information about complex medical conditions or treatments.

The current presentation of disclaimers, hidden behind an extra click and presented in a smaller font, fails to adequately address this risk. It assumes a level of skepticism and critical thinking that many users may not possess, especially when they are actively seeking answers to urgent health concerns. A more transparent and upfront approach to disclosing the limitations of AI-generated medical information is crucial to protecting public health and ensuring that individuals make informed decisions about their care.

while AI has the potential to be a valuable tool for accessing health information, it is essential that its limitations are clearly communicated to users. Prominent and easily visible disclaimers are a critical step in ensuring that individuals are empowered to make informed decisions about their health, rather than relying solely on potentially inaccurate or misleading AI-generated advice.