Microsoft AI Diagnostic System Outperforms Generalist Doctors in Complex Case Study

The race to integrate artificial intelligence into critical medical decision-making has accelerated with the publication of a significant new study in Science. The research, released on Thursday, offers the first large-scale comparison of a sophisticated AI diagnostic system against human generalist physicians in the context of complex, real-world clinical scenarios.

The system under scrutiny, known as MAI-DxO or DxO, functions as a coordinated panel of AI agents that mimic the collaborative behavior of a medical team. Rather than relying on a single model, DxO interrogates patient data, orders hypothetical tests, and debates potential diagnoses among its internal agents. In simulations using difficult cases from the New England Journal of Medicine, the system achieved an accuracy rate of 83%. By comparison, primary care doctors and general internists working individually with standard reference materials were correct in only 36% of instances.

Mustafa Suleyman, CEO of Microsoft AI, characterized the findings as a pivotal step toward what he terms "medical superintelligence." He noted that while specialists usually handle cases of this magnitude, the performance gap highlighted by the study is substantial. Dominic King, vice president of Health at Microsoft AI, emphasized that the core challenge in healthcare is the growing disparity between the supply of medical expertise and patient demand, a problem he believes AI is uniquely positioned to address.

The underlying mechanics of the system suggest that the majority of the performance gains stem from the power of the foundational large language models rather than the orchestration layer alone. Microsoft tested DxO with models from multiple providers, including OpenAI, Google, Anthropic, Meta, and DeepSeek. The highest results were achieved using OpenAI’s GPT-5 and GPT-5.5 models. When GPT-5 was used with a basic prompt, it scored 78.9% on complex cases at a cost of approximately $8,400 per case. The DxO layer reduced this cost by about $3,000 while adding roughly four percentage points to the accuracy score.

Cost and accuracy improvements varied depending on the model and the type of case. With the newer GPT-5.5 model, released in April 2026, the orchestration layer boosted accuracy from 72% to nearly 87% for complex cases at a slightly lower cost. However, for emergency department cases, the addition of DxO provided minimal benefit in either accuracy or cost efficiency, with both configurations scoring around 85%. For less capable models, such as Gemini 2.5 Flash, the boost was more pronounced, raising accuracy from 46.4% to nearly 69%, though the overall gap between weaker and stronger models remained larger than the specific gains provided by the orchestration system.

Microsoft is leveraging these findings to build out a broader infrastructure for healthcare AI. Suleyman revealed that the company is collaborating with the Mayo Clinic to train a dedicated healthcare foundation model, with a release planned for next year. He expressed confidence that this model will enable "really enormous leaps" in the field. Suleyman, who previously co-founded DeepMind and led its health initiatives before joining Microsoft in 2024, stated that he expects healthcare models to improve by an order of magnitude in capability within the next 24 to 36 months.

To validate the technology in real-world settings, Microsoft has partnered with Beth Israel Deaconess Medical Center in Boston. This upcoming study will evaluate DxO’s clinical reasoning across various hospital environments, including emergency and urgent care. Patients will receive standard treatment while consenting to parallel engagement with DxO. Researchers will then compare the AI’s recommendations against those of human physicians. King described this as potentially the largest study of its kind, aiming to provide compelling evidence for the large-scale deployment of such systems.

The publication of the study coincided with a manifesto from Microsoft outlining its vision for medical superintelligence, defined as the capacity to surpass human-led systems in medical reasoning and coordination across the entire arc of care. The company aims to move beyond simple benchmarking of individual tasks to deeper integration with clinical workflows. Suleyman noted that this focus on end-to-end care could serve as a differentiator for Microsoft compared to other frontier AI labs, which are also claiming that their systems can match or outperform human doctors.

Microsoft currently sells dictation and radiology tools to clinicians and views DxO as a natural expansion of its product lineup. Suleyman stated that the company is committed to building the top-tier medical clinical decision support products over the coming years. He predicted that by 2027, the industry will see a coding-style breakthrough in medical records, allowing for highly accurate predictions of how a patient’s condition might unfold, a capability he described as "gold dust" for clinicians.

As the study marks another milestone in the AI healthcare race, Microsoft joins Google and OpenAI in asserting that its technology is ready for rigorous real-world evaluation. The company’s leadership remains optimistic that the rapid iteration cycles seen in other AI domains, such as coding and chat, will soon translate into significant advancements in clinical decision support, ultimately reshaping the delivery of medical care.