Ontario AG finds flaws in AI scribes

TORONTO – A report released this week by Ontario’s auditor general has been making waves with findings of hallucinations and inaccuracies in the reporting done by AI scribes used in doctors’ offices. The critique of AI scribes was part of a wider study of AI usage across the provincial public service.

According to the special report, nine of 20 AI scribes fabricated information and made suggestions to patients’ treatment plans, such as referring the patient for therapy or ordering blood tests, even though these steps were not mentioned when the systems were tested through the use of simulated conversations between doctors and patients.

Moreover, 12 of 20 AI scribes evaluated captured a different drug than what was prescribed by the doctor.

And 17 of the 20 systems missed important details about the patients’ mental health issues, even though they were mentioned in the simulated recordings the AI scribes monitored.

The testing was done before 20 AI scribes were recommended to Ontario doctors in 2024.

The auditor general also criticized the testing and procurement process itself, a process designed by Supply Ontario in consultation with OntarioMD, Ontario Health and the Ministry of Health.

Eleven of the 20 approved vendors did not submit any third-party reports or ISO 27001 certification, as they were supposed to. Five vendors didn’t submit threat risk assessments or privacy impact assessments, as required, yet they were still approved.

Aware of the possible shortcomings of AI scribes, auditor general Shelley Spence (pictured) said she alerted her own doctor.

“I actually went to my doctor because you can hear that my voice isn’t exactly what it normally is … and they were using an AI scribe,” she said. “So, I kind of mentioned, ‘Please look at the transcript when you’re done.”

Critics of the report have noted some of its own flaws. Will Falk, a Toronto-based consultant who recently authored a CSA report on AI in Canadian healthcare, pointed out the evaluation in question was done several years ago – since then, new generations of AI systems have appeared, with higher quality.

“The Auditor General’s report headline that 2024-era scribes hallucinated is not news,” said Falk. “The industry has known that for several years, they probably still do. Every serious deployment assumes clinician review of AI generated notes.”

He added, “The AG report does not comment on current usage, it just comments on how the 2024 procurement was done. It and the Infoway program that followed were a rare case of government showing leadership. OMD, OMA, and the Ontario government should be congratulated on this program, not subject to scary headlines and attacks by ill-informed accountants.”

Falk said ambient scribes are implemented within a scope of practice as a complement, not a substitute, for clinician judgment. “28% of Canadian physicians now use AI scribes, three-quarters of them daily, and the College of Family Physicians of Canada is asking Ottawa to fund scribes for every family practice. The policy question is no longer whether ambient AI belongs in Canadian healthcare. It is whether we govern it formally or push clinicians back into unmanaged grey usage with consumer tools.”

Markham, Ontario family physician Dr. Paul Forman asserted that AI scribes have become important tools for doctors, reducing cognitive stress and burnout, but they must still be monitored.

“No matter how advanced a system is, no system can ever claim 100% accuracy or efficiency,” he said. “These technologies must always be used conservatively and scrutinized by the physician. At the end of the day, this is AI, and there are inherent risks whenever AI is involved in clinical environments. That is where it becomes a true patient-safety issue and why oversight is so critical.”

While AI scribes can relieve physicians of a great deal of note-taking and paperwork, Dr. Forman emphasizes that doctors must still check them for accuracy.

“I strongly believe these systems have tremendous value and can significantly augment physicians, improve workflow efficiency, reduce cognitive burden, and support safer, more guideline-aligned care. However, responsibility ultimately still rests with the physician. AI should be viewed as another clinical tool – whether an AI scribe or clinical decision support system – that can help make us more effective and informed, but never replace physician judgment, accountability, or careful review.”