A Science study found Microsoft's diagnostic system outperformed generalist doctors on difficult cases.
Why it matters: The results could shape clinical-AI deployment, liability debates and how vendors substantiate claims of medical superiority. But the system has not yet shown better patient outcomes or performance against specialist teams in live care.
- MAI-DxO scored 83% on 304 complex New England Journal of Medicine cases, versus 22% for doctors without references and 36% with references.
- The system scored about 80% on MIMIC-IV emergency-department cases, compared with 47% for participating physicians.
- A basic-prompt GPT-5 scored 78.9% on the complex cases, while adding DxO improved accuracy by roughly four percentage points and reduced estimated cost.
- Microsoft plans a prospective evaluation with Beth Israel Deaconess Medical Center in Boston alongside standard clinical care.
Microsoft researchers published a Science study on October 22, 2026, introducing the Sequential Diagnosis Benchmark and Microsoft AI Diagnostic Orchestrator, or MAI-DxO.
MAI-DxO coordinates multiple AI agents that propose diagnoses, request information, select tests, challenge hypotheses and weigh diagnostic value against estimated cost. The benchmark reveals case information only when the system or physician asks for it, then scores the final diagnosis and the sequence of requests.
On 304 complex clinicopathological cases derived from the New England Journal of Medicine, MAI-DxO reached 83% accuracy. Generalist physicians scored 22% without external references and 36% with textbooks and internet search. The physician comparison involved primary-care doctors and hospital generalists, rather than specialist teams that would typically handle unusually difficult cases.
The headline result also depends heavily on the underlying models. Reporting by TechCrunch found that basic-prompt GPT-5 scored 78.9% on the complex cases. DxO added roughly four percentage points while lowering estimated cost by about $3,000 per case. With GPT-5.5, accuracy rose from about 72% to nearly 87% when DxO was added, but the orchestration layer delivered little or no improvement on emergency-department cases.
Microsoft says a cost-optimized configuration maintained 81% accuracy at less than half the estimated cost of comparative systems. But the benchmark measures diagnosis, information requests and estimated costs, not patient outcomes, safety or treatment decisions. Independent clinical-AI research has also cautioned that static or text-heavy comparisons can overstate AI performance in real clinical environments.
Microsoft says the technology is ready for real-world evaluations. A planned study with Beth Israel Deaconess Medical Center, described by Reuters, will involve consenting patients in emergency-department and urgent-care settings while standard treatment continues.
By the numbers
- 83% - MAI-DxO accuracy on 304 complex NEJM-derived cases
- 22% - Generalist physician accuracy without external references on those cases
- 78.9% - Basic-prompt GPT-5 accuracy before adding the DxO orchestration layer
Yes, but: The strongest comparisons used retrospective, predominantly text-based cases and generalist physicians, not specialist teams. The study does not establish live-care safety, improved outcomes, calibration, false-positive or false-negative rates, or the system's legal and regulatory status.
What's next: Microsoft plans a prospective evaluation with Beth Israel Deaconess Medical Center in Boston, including emergency-department and urgent-care settings alongside standard treatment.