Microsoft Diagnostic AI Outperforms Physicians in Complex Cases
- Microsoft researchers published a study in Science on October 22, 2026, revealing an artificial intelligence diagnostic tool called MAI-DxO that achieved an 83.2% accuracy rate in evaluating complex...
- The system utilizes a mechanism called a diagnostic orchestrator to break down complex medical puzzles into sequential steps rather than analyzing all patient symptoms instantly.
- In test evaluations covering 304 complex and unusual clinical cases, the coordinated AI model achieved a diagnostic accuracy exceeding twice that of physicians consulting reference materials and the...
Microsoft researchers published a study in Science on October 22, 2026, revealing an artificial intelligence diagnostic tool called MAI-DxO that achieved an 83.2% accuracy rate in evaluating complex medical cases from the New England Journal of Medicine, compared to 36.1% for physicians using reference materials. The study, titled Sequential Diagnosis Using Language Models
, appeared in volume 394, issue 6822 of Science, on pages 423–428; the journal received it on April 4, 2026 and accepted it on August 10, 2026. In an announcement released the same day, Microsoft stated that the work is published in the AI in Medicine
issue of the journal.
Microsoft Diagnostic Tool Outperforms Physicians
The system utilizes a mechanism called a diagnostic orchestrator to break down complex medical puzzles into sequential steps rather than analyzing all patient symptoms instantly. Reuters reported on Friday that the new tool relies on a system called the coordinator, which simulates the collaboration of a team of medical experts to reach a diagnosis, rather than analyzing all symptoms and tests at once like most current medical AI models. The model simulates a collaborative team of clinical experts to determine what questions to ask, what physical exams to prioritize, and which diagnostic tests to order. The tool updated its working hypotheses continually as new test results came back from simulated patient interactions. In all models tested using limited baseline prompts, DxO coordination improved diagnostic accuracy by an average of 7.9 percentage points in conference cases and 4.4 percentage points in emergency department cases.
In test evaluations covering 304 complex and unusual clinical cases, the coordinated AI model achieved a diagnostic accuracy exceeding twice that of physicians consulting reference materials and the internet. The model runs on GPT-5, distributing tasks across multiple specialized agents. Tests using emergency department data sets from the MIMIC-IV-ED database also showed higher accuracy compared to human baseline reviews. In the accompanying physician study conducted on 56 cases not used in the engineering work, participants submitted 764 diagnoses without references and 662 diagnoses using references; average accuracy was 22.3% without references at an average cost of $3,049 per case, and 36.1% with references at an average cost of $4,725. The best single physician achieved an accuracy of 51.8%.
Cost Analysis and SDBench Evaluation Benchmarks
Specific cost structures evaluated in the study showed that the AI achieved 80.9% accuracy at a computed cost of $2,241 per case, and 79.6% accuracy at $1,303 in cost-restricted settings, while a high-accuracy configuration reached 84.5% precision at $8,945. By comparison, human clinicians in the study’s physician arm averaged 22.3% accuracy without reference books at a cost of $3,049 per case, and 36.1% accuracy with references at an average cost of $4,725. Microsoft stated that no other evaluated approach, human or AI-based, achieved higher accuracy and lower cost simultaneously than MAI-DxO.
To establish these comparisons, researchers built a benchmarking framework called SDBench. SDBench converts published clinical clinicopathological conference summaries and emergency admission histories into step-by-step interactive diagnostic challenges, converting 304 cases of published clinicopathological conferences from the New England Journal of Medicine and 325 emergency department-to-inpatient transition cases from the MIMIC-IV-ED dataset into step-by-step interactive diagnostic encounters. Microsoft plans to release this benchmarking tool as an open-source resource to assist independent evaluations of diagnostic AI models.
Company researchers emphasized that paper-based test scores do not guarantee equivalent performance in routine clinical practice, noting that measured advantages are amplified in controlled simulations that exclude uncertain patient history, incomplete documentation, rapport-building, longitudinal follow-up, and multimodal data like radiology scans and pathology slides, and that published journal cases may represent diagnostically difficult cases at a higher rate than reality. Dominic King, who works on healthcare AI at Microsoft, noted that standard AI evaluations rely on multiple-choice questions where AI systems easily score 100%, noting that real patients never arrive bearing organized medical summaries.
Mustafa Suleyman, head of the artificial intelligence division at Microsoft, acknowledged that the technology requires clinical validation. Microsoft plans to initiate prospective hospital trials through a partnership with Beth Israel Deaconess Medical Center in Boston, where patients will interact directly with the system while independent physicians evaluate its performance. The company also stated that it is bringing together health teams from across Microsoft divisions, including researchers from Microsoft Research and the teams behind Dragon Copilot and PowerScribe, into an integrated Microsoft AI effort, pointing to its previously announced collaboration with the Mayo Clinic under a June 2, 2026 announcement where both organizations work to develop an advanced healthcare AI model to be owned by Mayo Clinic and made available by Microsoft through Azure Foundry APIs.