Microsoft's AI Got Difficult Diagnoses Right in 83 Percent of Cases, GPs in 36 Percent: There Were No Specialists in the Comparison

Microsoft's artificial intelligence made the correct diagnosis in 83 percent of complex medical cases, while general practitioners and internists, working alone with medical literature in front of them, got it right in 36 percent. The study was published in the journal Science, and Mustafa Suleyman, CEO of Microsoft AI, called it a step towards "medical superintelligence". A big word for a result obtained in a simulation.

The system is called MAI-DxO and works like a virtual panel of doctors: several AI agents question the patient, order tests and argue with each other until they agree on a diagnosis. The cases are taken from the New England Journal of Medicine, from the ones published precisely because they are difficult. And here is the small print missing from the headlines: in practice such cases are handled by specialists and teams of doctors - and they weren't part of the comparison.

The study's own data shows something Microsoft doesn't put on the front page: most of the improvement comes from stronger language models, not from the "panel". The company tested the system with models from OpenAI, Google, Anthropic, Meta and DeepSeek, and the best results came from GPT-5 and GPT-5.5 - OpenAI's models, not Microsoft's. GPT-5 on its own, with a short and simple prompt, got 78.9 percent of the complex cases right, at a cost of around 8,400 dollars (around 7,200 euros) per case. With MAI-DxO the cost fell by around 3,000 dollars, but accuracy rose by only four points.

With the newer GPT-5.5, released in April 2026, the picture is mixed. On the complex cases, the system raised accuracy from 72 to almost 87 percent, at a somewhat lower cost. On the emergency cases - no difference at all: with or without MAI-DxO the result is around 85 percent. It helps most with weaker models: Gemini 2.5 Flash gets 46.4 percent on its own and almost 69 with the system. But even then, the gap between the weak and the strong model is bigger than what Microsoft's orchestration adds.

Suleyman doesn't hide how convinced he is. "We're just true believers," he says, and predicts health models will get better "by a whole order of magnitude" in the next 24 to 36 months. Microsoft is training its own foundation model for health with the Mayo Clinic, due out next year, and Suleyman claims that in 2027 medical records will be on the verge of a breakthrough like the one AI made in programming. Forecast or sales pitch? Microsoft already sells doctors dictation and radiology tools, and Suleyman says openly that in the next few years the company "has to have the absolute best clinical decision support products".

The real exam is still to come. With Beth Israel Deaconess hospital in Boston, Microsoft will compare the system's recommendations with doctors' decisions for patients who agree to take part: they will be treated normally, while the AI gives its own opinion in parallel. Dominic King, vice president of health at Microsoft AI, says it will be "the largest study of its kind". Google has already published a study of its AMIE system with the same hospital in The Lancet, and OpenAI makes similar claims about its models. The race is about who gets into the doctor's office first.

King says the biggest problem in health care today is that demand for medical knowledge is growing faster than the number of doctors. That's exactly where AI will sell most easily. But when the machine gets it wrong and a tired doctor relies on it - who will be held responsible: the hospital, the doctor or the company that sold the system?