AI Healthcare Regulation: Avoiding Bias & Discrimination
- A new commentary in the Journal of the Royal Society of Medicine suggests that current risk-based regulations for artificial intelligence (AI) in health care are insufficient to protect...
- While AI and machine learning can improve clinical accuracy, concerns persist about their potential for inaccuracy, lack of openness, and inherent biases.
- The AI Act categorizes medical AI as "high risk" and imposes strict controls.
Current AI healthcare regulations may be failing to safeguard patient rights, according to a new commentary.This is the primary takeaway. Experts warn that a risk-based approach to AI in healthcare coudl lead to discrimination and both over- and undertreatment.The authors highlight the need for patient-centered AI regulation, including the right to clarification, consent, and seeking second opinions, especially concerning AI-driven diagnoses. News Directory 3 keeps you abreast of the latest developments.The piece highlights evolving patient rights within generative AI driven systems within healthcare, and emphasizes the necessity for active involvement from all stakeholders.Discover what’s next in patient-focused AI regulations.
AI in Health Care Needs Patient-Centered Regulation to Avoid Discrimination
Updated June 25, 2025
A new commentary in the Journal of the Royal Society of Medicine suggests that current risk-based regulations for artificial intelligence (AI) in health care are insufficient to protect patients. These shortcomings could lead to both over- and undertreatment, as well as discrimination against specific patient groups, the authors argue.
While AI and machine learning can improve clinical accuracy, concerns persist about their potential for inaccuracy, lack of openness, and inherent biases. The authors contend that the European Union’s AI Act, passed in 2025, does not adequately address these issues.
The AI Act categorizes medical AI as “high risk” and imposes strict controls. However, the authors believe this risk-based approach overlooks individual patient preferences, the systemic and long-term effects of AI implementation, and the limited involvement of patients in regulatory processes.
“Patients have different values when it comes to accuracy, bias, or the role AI plays in their care,” saeid Thomas Ploug, professor of data and AI ethics at Aalborg University, Denmark, and lead author of the commentary. ”Regulation must move beyond system-level safety and account for individual rights and participation.”
The authors propose introducing patient rights related to AI-generated diagnoses or treatment plans. These rights should include the ability to:
- Request an clarification.
- Give or withdraw consent.
- Seek a second opinion.
- Refuse diagnosis or screening based on publicly available data without consent.
Without immediate engagement from health care stakeholders, including clinicians, regulators, and patient groups, these rights risk being overlooked as AI rapidly evolves in health care, the authors warn.
“AI is transforming health care, but it must not do so at the expense of patient autonomy and trust,” Ploug said. “It is indeed time to define the rights that will protect and empower patients in an AI-driven health system.”
What’s next
The researchers urge health care stakeholders to collaborate on defining and implementing patient rights in the context of AI-driven health care to ensure ethical and equitable use of these technologies.