Microsoft's AI beat doctors on the hardest medical cases. Read the fine print

Story of the week. An AI system just did something that would have sounded like science fiction five years ago: in a peer-reviewed study published in the journal Science, Microsoft's diagnostic system, MAI-DxO, correctly diagnosed far more of the hardest medical cases than practicing doctors did. It's a big moment. It's also easy to overread, so here is what the study found, what it didn't test, and what it means for you.

What Microsoft built

MAI-DxO (short for Microsoft AI Diagnostic Orchestrator) isn't a single chatbot. It runs several AI "roles" that work like a panel of doctors: one proposes possible diagnoses, others challenge them, choose which tests to order and keep an eye on cost. Microsoft first described it in a 2025 preprint. The new version, reported by The Next Web, runs on OpenAI's GPT-5, and the earlier work showed it also improved results with other companies' models.

The results

The team tested it on 304 of the toughest case reports published in the New England Journal of Medicine, the kind doctors read as puzzles. Generalist physicians took the same cases and were allowed to use textbooks and the internet.

A second test used 325 real emergency department admissions, and the gap held: 80.3% for the system against 46.8% for doctors. Microsoft also said the system's estimated testing costs were lower, though the report didn't give dollar figures.

Microsoft's Dominic King made the point that AI now aces multiple-choice medical exams, saying it has become "very good at getting a 100% pass rate on those." That's why the team moved to messy, step-by-step cases instead.

The fine print

• Text only. The system and the doctors worked from written case details. No one examined a patient, looked at a scan or talked to a family. Real medicine involves all of that.
• Generalists, not specialists. The doctors were generalists. Many of these cases would normally go to a specialist.
• Not in use. MAI-DxO isn't treating anyone. Microsoft's next step is testing it for triage with Beth Israel Deaconess Medical Center in Boston.
• Benchmarks aren't bedsides. Published case reports are chosen because they are unusual. Everyday care is mostly common problems, where the gap may look very different.

What it means for you

• Don't replace your doctor with a chatbot. Consumer AI apps aren't MAI-DxO, and even MAI-DxO isn't approved for patient care. If something feels wrong, see a clinician, and in an emergency call your local emergency number.
• Do use AI to prepare. A good use today is to organize your symptoms, timeline and questions before an appointment, then take that list to your doctor. You'll get more out of a short visit.
• Expect AI to show up behind the scenes first. Triage, second opinions and paperwork are where hospitals are likely to adopt it, with doctors checking the results.
• Watch for real-world trials. The question that matters is not "can AI pass the test?" but "do patients do better?" That's what the hospital trials will start to answer.

This article is general information, not medical advice.

Disclosure: this site's posts are drafted with AI help. This one was drafted with Claude, made by Anthropic, a competitor of OpenAI and Microsoft. It sticks to published reporting.

Sources

• Report on Microsoft's MAI-DxO study in Science, The Next Web, Oct 23, 2026
• The Path to Medical Superintelligence, Microsoft AI (background on MAI-DxO's design, 2025)