Microsoft AI Hits 83% Accuracy on Complex Cases, Outpacing Human Doctors in Study
A new multi-agent system demonstrates superior diagnostic accuracy compared to generalist physicians in a simulation of difficult medical cases, though cost and model dependencies remain key variables.
By Sarah Montgomery ·
83 percent. That is the accuracy rate achieved by Microsoft’s new AI diagnostic system, MAI-DxO, when tasked with solving complex medical cases in a simulation, a figure published in Science this week. The system, which coordinates multiple AI agents to mimic a panel of physicians, correctly identified diagnoses in these difficult scenarios more than twice as often as generalist doctors. While the results position the technology as a significant step toward what Microsoft CEO Mustafa Suleyman calls “medical superintelligence,” the study reveals that performance is heavily dependent on the underlying models used and the specific complexity of the case.
The core of the system is a collaborative framework where multiple AI agents question patients, order tests, and debate potential diagnoses. In the simulation, which utilized complex cases from the New England Journal of Medicine, this orchestrated approach proved far more effective than individual human effort. Primary care physicians and general internists, working alone with reference materials, were correct only 36 percent of the time. However, a crucial caveat exists: these highly complex cases are typically handled by specialists or team-based care in real-world settings, groups that were not included in this specific comparison.
Dominic King, vice president of Health at Microsoft AI, emphasized the practical urgency of the technology, noting that the primary challenge in healthcare is the widening gap between medical expertise supply and patient demand. By publishing these findings, Microsoft joins rivals Google and OpenAI in asserting that AI can match or exceed human diagnostic capabilities. Yet, the data suggests that the bulk of the accuracy gains stems from more powerful base models, with the DxO orchestration layer providing incremental improvements on top.
Cost and model selection significantly influence outcomes. When using OpenAI’s GPT-5, DxO reduced the per-case cost by approximately $3,000 compared to running the model alone, while increasing accuracy by only four points. With the newer GPT-5.5, released in April 2026, the system boosted accuracy from 72 percent to nearly 87 percent for complex cases at a slightly lower cost. However, for emergency department cases, the gains were minimal, with or without the orchestration layer. Weaker models, such as Gemini 2.5 Flash, saw a more pronounced boost from DxO, rising from 46.4 percent to nearly 69 percent, though the gap between strong and weak models remained larger than the boost provided by the system.
Suleyman stated that Microsoft is collaborating with the Mayo Clinic to train a dedicated healthcare foundation model, with a release planned for next year. He predicts that by 2027, the company will be close to a breakthrough in processing medical records, allowing for highly accurate predictions of patient outcomes. This foresight, he argues, is invaluable for clinicians. To validate the technology, Microsoft has partnered with Beth Israel Deaconess Medical Center in Boston to evaluate DxO in real-world hospital settings, comparing its recommendations against those of human doctors in both emergency and urgent care environments.
The study coincides with a broader manifesto from Microsoft on building medical superintelligence, defined as the ability to surpass human-led systems in reasoning and coordination across the full arc of care. King described the upcoming real-world evaluation as potentially the largest of its kind, aiming to provide compelling evidence for large-scale deployment. Suleyman remains confident that generative AI in healthcare will improve by an order of magnitude within the next 24 to 36 months, driven by a “flywheel” effect where real-world usage identifies gaps and drives rapid model iteration.
As Microsoft expands its existing portfolio of dictation and radiology tools, Suleyman is determined to secure the top position in medical clinical decision support. He views this focus as a key differentiator for the company among frontier AI labs. “We’re just true believers,” Suleyman said, underscoring the company’s commitment to accelerating the introduction of these advanced diagnostic tools into the global healthcare landscape.
