Microsoft's Diagnostic AI Outperforms General Practitioners in Scientific Study

Microsoft’s MAI-DxO diagnostic system outperformed participating generalist physicians on complex diagnostic cases in a peer-reviewed study published October 22, 2026, in Science, which reported 83.2% accuracy for the system on New England Journal of Medicine case records versus 36.1% for physicians using reference resources.

The study, “Sequential Diagnosis with Language Patterns,” appears in Science Volume 394, Number 6822, pages 423–428; was received April 4, 2026, and accepted August 10, 2026. All 15 authors are Microsoft employees, and the list includes Mustafa Suleyman, CEO of Microsoft AI, along with Dominic King and Eric Horvitz. In an announcement the same day, Microsoft said the work “is published today in Science” in the journal’s “AI in Medicine” issue.

A reference point built around step-by-step diagnosis

The article introduces the Sequential Diagnosis Benchmark, or SDBench, which converts 304 clinical pathology conference cases from the New England Journal of Medicine and 325 emergency room to hospitalization encounters from the MIMIC-IV-ED dataset into interactive, stepwise diagnostic encounters. Each case opens with a short clinical vignette, from which the diagnostician, be it a doctor or an AI system, can ask questions about the history or exam, order diagnostic tests, or commit to a final diagnosis. Performance is evaluated based on diagnostic accuracy compared to the reference diagnosis and the estimated cumulative cost of tests ordered.

Two support agents structure each meeting. A Gatekeeper agent mediates each information request against the underlying case documentation and generates diagnostically neutral synthetic results when a requested result is absent, so missing data does not signal the answer. An agent judge evaluates the assigned diagnosis against the reference diagnosis on a five-level rubric written by the doctor, with scores of 4 or 5, representing diagnoses for which management would remain unchanged, counted as correct. Test costs were estimated using CPT-based prices derived from the Kaiser Permanente Washington Central Hospital standard cost file, with medical history and examination questions built into a fixed cost for the physician visit.

As part of the benchmark, the authors evaluated Microsoft AI Diagnostic Orchestrator, abbreviated DxO in the paper, a model-agnostic system that simulates a panel of clinical experts with complementary roles, applies cost-conscious test management at each stage, and aggregates diagnoses across independent runs. Microsoft’s announcement describes the system as a group of virtual doctors debating in several rounds. The authors argue that static benchmarks, which present all case information upfront and require a single response, cannot measure whether a system asks the right questions, avoids indiscriminate testing, or recognizes when it has enough certainty to conclude a workup.

Results on NEJM and Emergency Room cases

In clinical pathology conference cases, DxO paired with GPT-5 and operated without budget constraints achieved an accuracy of 83.2% at an average cost of $5,431 per case. Cost-constrained configurations achieved 80.9% accuracy at $2,241 and 79.6% at $1,303, while a configuration targeting diagnostic excellence achieved 84.5% at $8,945. In the companion physician study, conducted on 56 cases excluded from engineering work, participants produced 764 closed-book and 662 open-book diagnoses, with an average accuracy of 22.3% closed-book at $3,049 per case and 36.1% open-book at $4,725. The best single doctor achieved 51.8%. The article also reports different testing styles: Doctors asked an average of 5.6 questions before ordering their first test, while DxO with GPT-5 asked an average of 25.5, a strategy the authors describe as extensive structured history taking.

In emergency room cases, DxO without budget constraints achieved an accuracy of 80.3% at $4,979 per case and 73.2% at $2,634 with budget constraints. Generalist doctors averaged 46.8% at $3,400, while the best individual doctor achieved 62.5%. Across all models tested with minimal background instruction, DxO orchestration increased diagnostic accuracy by an average of 7.9 percentage points on conference cases and 4.4 points on emergency room cases. Microsoft’s announcement claims that no other human or AI-based approach has been both more accurate and less expensive than MAI-DxO, describing the system as 1.7 times more accurate than doctors in emergency room presentations, more than double that of doctors using textbooks and the Internet on journal cases, and nearly four times more accurate than doctors diagnosing without such resources, at half to a third of the cost.

DxO was initially developed when GPT-4.1 was the most powerful model available, was subsequently evaluated with GPT-5, and the entire evaluation suite was repeated with GPT-5.5 before publication without changing the prompts or orchestration, with an accuracy advantage over the baseline prompt increasing. An earlier version of the work was published as a preprint on June 27, 2025, reporting 80% accuracy with OpenAI’s o3 model, four times the clinician average of 20%, and 85.5% in a highest accuracy setting; The peer-reviewed version adds the emergency room case set and updated assessments with GPT-5 and GPT-5.5.

Author’s stated limitations and disclosures

The authors say the comparisons show no superiority over specialists, multidisciplinary teams, or real-world clinical practice, and write that they would expect the performance gap to narrow compared to expert specialists or teams matched to each case. They note that measured benefits are amplified within a controlled simulation that sets aside uncertain patient histories, incomplete documentation, reports, longitudinal follow-up, and multimodal data such as imaging and pathology slides, and that journal cases may overrepresent diagnostically challenging presentations. A further stated limitation is comparison with individual physicians rather than multispecialty collaboration, and the Judge’s rubric evaluates diagnostic accuracy at the disease level rather than therapeutic appropriateness, safety, or longitudinal management.

In the document disclosures, the authors state that they are all Microsoft employees who hold equity in the company, that many are inventors on a Microsoft patent application related to an automated scoreboard for sequential diagnosis, and that no funds external to Microsoft were acquired for the studies.

Open source release and clinical partnerships

Microsoft said it is making SDBench open source for select NEJM cases, including benchmarks, Gatekeeper, Judge, and cost estimator code. The code is available on GitHub under the MIT license, with a configuration called “science” that blocks the gatekeeper and judge used for the document’s results. According to the SDBench project site, the dataset derived from the journal cases requires licensing from the NEJM Group, the MIMIC subset is distributed via PhysioNet subject to credentialing, and five fully synthetic cases are provided as the default open dataset.

A position paper, “Medical Superintelligence: Foundations and Aspirations,” introduces six interdependent dimensions of medical intelligence and states that crossing the threshold of medical superintelligence will require “more than superhuman performance on isolated tasks,” that is, systems that surpass today’s strongest human-driven healthcare systems in integrated medical reasoning across the entire spectrum of care. Microsoft’s announcement states that the article was published in BMJ Digital Health and AI. The company also said it is engaging healthcare teams from across Microsoft, including scientists from Microsoft Research and the teams behind Dragon Copilot and PowerScribe, in an integrated effort at Microsoft AI, and referenced its previously announced collaboration with the Mayo Clinic; with an announcement on June 2, 2026, the organizations are developing a frontier AI model for healthcare that Mayo Clinic will own and that Microsoft plans to make available through Azure Foundry APIs.

Through a new partnership with Beth Israel Deaconess Medical Center in Boston, Microsoft said it will evaluate clinical reasoning skills in hospital settings in a prospective study where patients interact directly with the system and performance will be measured against doctors’ independent assessments.