Microsoft AI vs Doctors:
MAI-DxO Diagnosis Study and 83% Accuracy Explained

ONE PATIENT, TWO REASONING PATHS

The clue was not hidden in a scan. It was in the patient's story.

A patient had been falling for months. There was weakness in the legs, unexplained bruising, bleeding gums and lost teeth. A previous gastric bypass and a diet poor in fruit provided another clue. It was the kind of presentation where several serious explanations could seem reasonable.

In a case described by the Microsoft researchers, a directly prompted GPT-5 model pursued a broad series of neurological and other investigations. It missed the decisive diagnosis. MAI-DxO, using the same base model but a different reasoning process, connected the bleeding and dietary history to scurvy, then selected a plasma vitamin C test. The researchers estimated the model's workup at $8,806 versus $1,161 for the orchestrated approach, which reached the recorded diagnosis.

This is a research example reconstructed from the authors' report, not a new patient interview or a real-time hospital encounter. It does not prove that a deployed system would behave the same way with a living patient. But it captures the paper's central question: could AI improve diagnosis not by knowing more answers, but by choosing better questions? Details: authors' detailed manuscript, results discussion.

THE NEWS / OCTOBER 22, 2026

What Microsoft announced — and why Science matters

On October 22, 2026, researchers from Microsoft published Sequential diagnosis with language models in Science (DOI: 10.1126/science.aeh7930). An earlier version of the work circulated in 2025; the October publication brought the sequential-diagnosis results into a peer-reviewed journal. The team's work is notable for examining the path to a diagnosis rather than presenting an AI system with the full case and asking for a multiple-choice answer.

The company calls its system MAI-DxO, short for Microsoft AI Diagnostic Orchestrator. It is not a newly trained medical foundation model or a single chatbot with a special medical prompt. It is a reasoning workflow arranged around foundation models, virtual specialist roles, a controlled information interface and a cost calculation. That difference is the story behind the press headline. The peer-reviewed Science paper, Microsoft AI's October 22 announcement and PubMed record establish the publication and results.

INSIDE THE SYSTEM

From a symptom to a diagnosis: a process, not a one-shot answer

Think of the traditional medical-AI quiz: the model sees the patient's history, examination, imaging and laboratory results all at once, then chooses a diagnosis. Real clinical reasoning rarely works like that. A doctor starts with incomplete information. Each new answer can make the next test more valuable — or less necessary.

The benchmark, SDBench, supplies the surrounding simulation. A language-model Gatekeeper controls what information becomes visible. A separate Judge grades the diagnosis using a physician-designed rubric. A Cost Estimator assigns standardized estimated prices to the requested workup. Meanwhile, MAI-DxO coordinates a virtual panel with roles such as hypothesis tracking, skeptical challenge and resource stewardship. The “panel” is AI-simulated reasoning roles, not a team of independently practicing doctors.

That architecture matters. It offers a possible way to reduce anchoring: the tendency to settle on an appealing first diagnosis and seek evidence that supports it. Yet multiple agents do not automatically create independence or safety. Their judgments can share biases because they run on related underlying models and information. Full design details appear in the authors' detailed manuscript and the Microsoft AI's October 22 announcement.

TWO DATASETS / TWO CLAIMS

The results: 83.2% on complex teaching cases, 80.3% on emergency records

The study used two distinct sets of retrospective cases: 304 difficult clinicopathological conference cases published in the New England Journal of Medicine and 325 emergency-department-to-admission cases from MIMIC-IV-ED. The second set is closer to routine hospital documentation, although it still consists of selected admissions presented through a simulated interface.

That is roughly 2.3 times the reported physician accuracy on the NEJM task and 1.7 times on the emergency dataset. But these are ratios of success rates, not “2.3 times better doctors” in everyday practice. Clinical skill includes examining patients, managing uncertainty, discussing treatment options, communicating risk and following outcomes over time. The peer-reviewed Science paper addresses only a narrower diagnostic task.

WHAT THE HEADLINE LEAVES OUT

Were the doctors and the AI tested on equal terms?

The comparison contains thoughtful design choices — and important asymmetries. The authors recruited practicing generalist physicians, with a median of 12 years' experience in both initial physician groups. In the NEJM arm, 21 physicians participated without external references and 19 different physicians participated with access to non-generative medical references including UpToDate and clinical guidelines. A further 16 physicians completed the emergency evaluation. Generative-AI assistance was not available to the physicians.

Crucially, the NEJM physician experiment used 56 held-out cases, while the widely quoted MAI-DxO figure is reported for the full 304-case collection; the authors also examined performance on the held-out subset to assess generalization. The headline percentages should not be mistaken for every person having evaluated exactly the same 304 cases. The human study produced multiple diagnoses per case because physicians worked through overlapping subsets, and the physician figures represent averages.

The comparator was an individual generalist, not a multidisciplinary team of infectious-disease experts, oncologists, radiologists and other specialists. That distinction matters particularly for NEJM teaching cases selected because they are unusual and difficult. At the same time, the emergency set strengthens the paper because the pattern of advantage was not confined to rare educational puzzles.

These limitations do not erase the result. They define what the result means. For audit details, including the physician groups, held-out split and test interface, see the authors' detailed manuscript, methodology and results.

THE ECONOMICS / A CONDITIONAL ADVANTAGE

Was Microsoft's AI cheaper than doctors? Not in every configuration.

Cost is where the usual summary can become misleading. On the complex NEJM cases, the authors report $5,430 in average estimated tests for unconstrained MAI-DxO built on GPT-5, compared with $8,409 for directly prompted GPT-5. This is a meaningful improvement in both modeled spending and diagnostic accuracy over the un-orchestrated model.

But the open-book physician group averaged $4,725 in estimated tests. In that particular standard configuration, MAI-DxO was more expensive than the human comparator, even though it was much more accurate. Changing the AI's cost constraint changes the picture: the researchers describe a configuration with 80.9% accuracy at $2,241, and another with 79.6% at $1,302 on the NEJM cases. Accuracy and cost have to be discussed together, using the same configuration.

The operational insight may matter more than the dollar figure: an AI system that evaluates the likely information gained from a question or test could support a more disciplined diagnostic workup. Whether it actually reduces spending or unnecessary testing in a real hospital remains unproven.

WHY THIS RESEARCH IS SCIENTIFICALLY USEFUL

The new benchmark measures the search for evidence, not just the final guess

Traditional medical question-answering evaluations often reveal all relevant facts in advance. SDBench instead makes the diagnostic agent request information step by step. It asks three different questions: Did the system reach the recorded diagnosis? What evidence did it request to get there? What was the estimated cost?

That shift is genuinely useful for evaluating AI agents. It exposes systems that guess too early, order expensive tests indiscriminately or cannot update their working hypotheses when new information contradicts their original view. It also makes comparisons across model families possible: the authors report that their orchestration methods improved multiple underlying model families, although the magnitude varied.

Researchers can inspect the released SDBench resources for selected NEJM cases through Microsoft's benchmark project. The value of a public evaluation framework depends on careful external replication, dataset governance and reporting of model versions and test settings, not just on one impressive bar chart.

THE MOST IMPORTANT SCIENTIFIC CAVEATS

A simulated patient answers differently from a real patient

Both physicians and AI interacted with a text-only Gatekeeper, not a person in front of them. That controls the experiment, but it strips away real clinical signals: the hesitations in a patient's story, incomplete records, conflicting measurements, visual inspection of an image, physical findings and changes over hours or days.

The Gatekeeper could also generate plausible synthetic findings when the original record lacked a requested detail. The research team evaluated its behavior, but synthetic answers still mean that parts of the interaction were a model-generated approximation rather than an observed medical fact. This issue was especially prominent in the shorter and less standardized emergency records.

Neither dataset sampled healthy people or ordinary benign presentations, so the disease mix does not represent routine primary care. A system could be skilled at resolving difficult cases and still order inappropriate investigations when a patient is probably healthy. The study also focused on a final diagnostic label, not on safe treatment choices, communication, longitudinal follow-up or whether a patient's health improved.

There are also open questions for external validation: whether future systems maintain performance on unseen hospitals, languages and patient populations; whether the automated judge agrees with independent clinical panels in difficult disagreements; and whether gains persist against specialists working as teams. The study's authors explicitly identify the simulation and individual-physician comparison as limitations in the authors' detailed manuscript.

THE LARGER AI STORY

This may be as much an AI-agent architecture result as a medical-model result

MAI-DxO's most interesting contribution is its approach to orchestration. Rather than assuming a larger model will always perform better when asked one question, the system assigns complementary cognitive jobs: propose, challenge, investigate, estimate cost and revise. The authors report gains when the orchestration is paired with several model families, including models introduced after the original system was designed.

That is relevant beyond healthcare. In software engineering, an agent can test a hypothesis before changing code. In scientific research, it can request the experiment that most reduces uncertainty. In medical diagnosis, those same ideas confront much higher safety stakes. More steps and more virtual agents are not themselves proof of reliability; each additional stage needs independent validation, traceable evidence and a clear way to stop or escalate.

For the framework behind those concerns, read SXF's AI Agent Reliability research, guide to AI model benchmarks and introduction to agentic AI. MAI-DxO is an unusually consequential case study in the difference between answer accuracy and system behavior.

FROM PAPER TO PRACTICE

The next meaningful test is not another record-breaking benchmark

Microsoft has announced plans to collaborate with Beth Israel Deaconess Medical Center in Boston on prospective evaluations across hospital settings. Its announcement describes patient interaction and comparisons against clinicians' independent assessments. These plans are not published proof of successful clinical deployment; those results have yet to be demonstrated in the sources reviewed here.

The questions for prospective evaluation are practical and measurable: Does the system help clinicians catch otherwise missed conditions? How often does it recommend unnecessary or unsafe investigations? Can it recognize uncertainty and defer appropriately? How does it handle incomplete data, demographic differences, privacy requirements and audit trails? Does it improve patient outcomes rather than just the percentage of cases with the right label?

SXF's assessment: The study provides credible evidence that sequential, cost-aware AI orchestration can outperform single generalist physicians on the tested written diagnostic tasks. It does not justify a claim that an autonomous AI doctor is clinically ready. The most promising near-term hypothesis is carefully governed clinical decision support — an additional source of diagnostic possibilities and evidence checks under physician oversight — subject to prospective safety and effectiveness testing.

FREQUENT QUESTIONS

What readers should know about Microsoft's AI diagnostic study

What is Microsoft MAI-DxO?

MAI-DxO is a diagnostic orchestration system. It uses a foundation model with virtual clinical reasoning roles and iterative access to case information, rather than producing a diagnosis from one complete prompt.

Did it really beat doctors?

It scored higher than participating generalist physicians on the two controlled benchmark datasets reported in Science. The experiment did not compare it against specialist teams caring for actual patients.

How many medical cases did Microsoft test?

The benchmark contained 304 complex NEJM teaching cases and 325 emergency-to-admission cases. The NEJM physician comparison used a held-out 56-case subset, not all 304.

Was the AI cheaper?

It was cheaper than directly prompted GPT-5 in the cited unconstrained NEJM comparison; that same setting was not cheaper than the open-book physician group's estimated test spending. Cost-controlled variants were less expensive in the simulation.

Can I use this system for my own medical diagnosis?

This publication is not evidence that the research system is safe or available for personal clinical use. Symptoms and treatment decisions need appropriate licensed clinical care, particularly for urgent or worsening conditions.

SOURCE TRANSPARENCY

Read the original research, not just the headline

This is SXF editorial analysis of a peer-reviewed study and the authors' related technical materials. SXF did not run the benchmark or independently test the diagnostic system. The paper's accuracy and cost measures are reported findings, not SXF-generated experimental results.

Editorial methodology: source links are provided adjacent to central claims; differences between simulations and clinical care are stated explicitly. This page offers technology research analysis, not medical advice. How SXF evaluates sources →