A Microsoft AI Panel Outdiagnosed Doctors on Hard Cases at About Half the Cost

A panel, not a single answer

Microsoft has published initial results for MAI DxO, short for Microsoft AI Diagnostic Orchestrator. The Neuron describes it as a virtual panel of doctors: AI agents propose diagnoses, order tests, and challenge each other as new results come in, over several rounds, while weighing what each test costs so the panel cannot simply order everything.

The Rundown AI covered the same release the same morning, and both newsletters land on the same headline finding: the panel beat physicians on hard cases and on everyday cases, while spending less on testing.

The scores, stated plainly

The test used SDBench, a sequential diagnosis benchmark Microsoft built because real diagnosis never hands you the whole case at once. The player starts with limited information and decides what to ask or order next, one step at a time. It covers 304 hard cases from the New England Journal of Medicine plus more typical emergency room cases. The results:

• Hard journal cases: MAI DxO solved 83 percent, versus 36 percent for physicians with textbooks and internet access and 22 percent without
• Typical emergency room cases: MAI DxO hit 80 percent, versus 47 percent for physicians
• Cost: Microsoft says the panel reached the right diagnosis at roughly half to a third of the cost in the journal work, ordering fewer tests

The fine print matters as much as the score

The Neuron is careful with the caveats, and they deserve equal billing. The doctors in the test were primary care physicians and general internists working alone, not the specialists and teams who usually handle cases this hard. It was a simulation with no real patients, and Microsoft says real world performance is still to be tested.

Most of the gain also comes from the models underneath. The Neuron notes GPT 5 alone scored 78.9 percent on the hard cases, and the panel structure added about four points on top. The panel is an improvement on a strong model, not magic layered over a weak one.

The real test is in Boston

What comes next decides whether this is a benchmark story or a medicine story. Microsoft will test the system in real clinical settings at Beth Israel Deaconess Medical Center in Boston, where consenting patients also talk to the system and researchers compare its recommendations with the human doctors assessments.

Microsoft also plans a healthcare model with Mayo Clinic next year. There is no pricing or availability for MAI DxO itself yet. The number to watch is how often the panel agrees with real emergency doctors when real patients are in the room.

The panel idea is the part worth stealing

Strip away the medicine and a general lesson remains for anyone who works with AI. A single model answering once is a first draft. Several agents proposing, questioning, and challenging each other before answering produced a measurably better result, at lower cost, because weak ideas got argued down before they became the answer.

That pattern travels. Use it on a script, a price, or a plan before money moves:

• Ask for two or three independent takes before you let any one of them see the others
• Give one agent the explicit job of arguing against the leading answer
• Make cost a scored input, not an afterthought, the way the panel weighs each test it orders
• Judge the panel on real work over time, not on one impressive demonstration

Diagnosis is one step, not the whole job

The Rundown adds the closing caution. Getting the assessment right is only one step in care. Helping patients still requires guiding treatment, tracking health over time, and fitting into how doctors and patients decide together, all of which has to be proven at scale.

That humility is the right frame for every AI headline like this one. The score is real, the setting was simulated, and the honest position is both at once.