Ontario’s AI medical scribes fail 60% of drug tests, exposing procurement disaster
Government-approved systems deployed to 5,000 physicians recorded wrong medications in majority of simulated cases, with accuracy weighted at just 4% in vendor selection.
Ontario’s Auditor General found that 60% of government-approved AI medical scribes recorded a different drug than what physicians prescribed in controlled tests, revealing catastrophic accuracy failures in systems now used by thousands of doctors across the province.
The audit, released this week by Auditor General Shelley Spence, exposes a procurement process that prioritised vendor location over clinical safety. Accuracy of medical notes accounted for just 4% of evaluation criteria, while domestic presence in Ontario was weighted at 30%—nearly eight times higher. Approximately 5,000 Ontario physicians currently use these systems, according to the CBC, with no mandatory requirement for doctors to verify AI-generated notes before they enter patient records.
60%
85%
45%
4%
Hallucinations and fabricated treatment plans
Beyond drug identification failures, 17 of 20 approved AI scribes missed critical details about patients’ mental health in at least one of two test scenarios, per the Office of the Auditor General. Nine systems fabricated information entirely, generating treatment suggestions that were never mentioned in simulated doctor-patient conversations.
The testing methodology itself was compromised. Vendors submitted systems via offline evaluations where they could overstate compliance, and 11 of 20 approved vendors failed to provide third-party security audits. Five vendors did not submit threat risk assessments or privacy impact assessments yet were approved for deployment. The province launched the AI scribe programme in April 2025 despite these validation gaps.
“Inaccuracies in medical notes generated by AI Scribe systems could potentially result in inadequate or harmful treatment plans that may potentially impact patient health outcomes.”
— Shelley Spence, Ontario Auditor General
Procurement weighted location over accuracy
The evaluation framework reveals how clinical safety became subordinate to economic policy. Domestic vendor presence received 30% of total points, while accuracy of generated medical notes—the core function of the technology—accounted for 4%. The Register reports that this weighting allowed systems with documented reliability failures to pass procurement thresholds based on vendor geography rather than clinical performance.
Ontario Minister of Public and Business Service Delivery Stephen Crawford defended the programme’s intent, stating the technology gives physicians “more time to spend with their patients and less time in record keeping.” That efficiency claim now faces scrutiny given the error rates documented in the audit.
Regulatory gap exposed
The audit arrives amid a broader regulatory debate over AI deployment in clinical settings. Health Canada released pre-market guidance for machine learning-enabled medical devices in February 2025, requiring manufacturers to demonstrate safety and effectiveness through lifecycle evidence. Ontario’s AI scribe procurement predated full implementation of these standards.
Current usage is widespread. Twenty-eight percent of Canadian physicians now use AI scribes, with three-quarters deploying them daily, according to Canadian Healthcare Technology. The College of Family Physicians of Canada has requested federal funding to provide scribes for every family practice—a proposal now complicated by documented accuracy failures.
Ontario’s Information and Privacy Commissioner released guidance on AI scribes in January 2026, requiring human oversight and accuracy procedures under provincial health privacy legislation. The Auditor General’s findings suggest these safeguards were not adequately enforced during initial deployment.
Liability and accountability questions
The absence of mandatory physician sign-off on AI-generated notes creates ambiguity around clinical responsibility. Doctors using the approved systems were not required to formally attest that notes were accurate before they became part of patient records. Dr. Paul Forman, a Markham family physician, told Canadian Healthcare Technology that “no system can ever claim 100% accuracy or efficiency. These technologies must always be used conservatively and scrutinized by the physician.”
That scrutiny appears to have been missing at the procurement stage. NDP Leader Marit Stiles responded to the audit findings by stating, “There do not seem to be any guardrails around this. So it’s very concerning.” Supply Ontario has now agreed to determine the feasibility of including mandatory note confirmation and yearly external audits in future AI scribe contracts.
- 60% drug identification error rate establishes concrete benchmark for unacceptable AI performance in clinical documentation.
- Procurement methodology that weighted vendor location 7.5x higher than accuracy sets precedent for what not to replicate in healthcare AI procurement.
- Absence of mandatory physician verification creates liability exposure for both healthcare providers and provincial health authorities.
- Widespread adoption (28% of Canadian physicians) means remediation affects thousands of active deployments, not theoretical future rollouts.
What to watch
Health Canada’s response will determine whether this triggers federal intervention in provincial health AI procurement. The agency’s machine learning device guidance establishes standards that Ontario’s process violated—watch for enforcement action or mandatory compliance timelines.
Provincial medical colleges face pressure to issue updated practice standards. The audit exposes a gap between existing physician oversight obligations and the technological reality of AI-generated clinical documentation. Formal guidance on verification requirements and liability allocation is likely within 90 days.
Vendor accountability remains undefined. The 20 approved systems failed at documented rates, yet no vendor has been suspended or delisted. Whether Ontario pursues contract remediation or maintains deployment of defective systems will signal how seriously regulators treat AI safety failures in regulated healthcare environments.
Class action risk is elevated. Patients whose treatment was guided by inaccurate AI-generated notes have potential standing for negligence claims. Watch for legal filings targeting both healthcare providers and the provincial procurement authority within six months.