Siri, Alexa not as good as dialling 911

EDMONTON – Turning to virtual assistants for medical emergency information can be useless, shows new research from the University of Alberta. “We were hoping to find that the devices would have a better response rate, especially to statements like ‘someone is dying’ and ‘I want to die,’ versus things like ‘I have a sunburn or a sliver,’” said lead author Christopher Picard (pictured), a master’s student in the Faculty of Nursing and a clinical educator at Edmonton’s Misericordia Community Hospital emergency department.

“I don’t feel any of the devices did as well as I would have liked, although some of the devices did better than others.” Google Home and Alexa are more reliable than Siri and Cortana, researchers found.

About 50 per cent of internet searches will be voice-activated by the end of 2020, added co-author Matthew Douma, assistant adjunct professor in critical care medicine.

The researchers tested four commonly used devices – Alexa, Google Home, Siri and Cortana – using 123 questions about 39 first aid topics from the Canadian Red Cross Comprehensive Guide for First Aid, including heart attacks, poisoning, nosebleeds and slivers.

The devices’ responses were analyzed for accuracy of topic recognition, detection of the severity of the emergency in terms of threat to life, complexity of language used and how closely the advice given fit with accepted first aid treatment guidelines.

Google Home performed the best, recognizing topics with 98 per cent accuracy and providing advice congruent with guidelines 56 per cent of the time. Google’s response complexity was rated at Grade 8 level.

Alexa recognized 92 per cent of the topics and gave accepted advice 19 per cent of the time at an average Grade 10 level.

The quality of responses from Cortana and Siri was so low that the researchers determined they could not analyze them.

Picard, an emergency room nurse, was inspired to launch the study, wondering whether there might be a use for virtual assistants during a medical emergency.

“The best example of hands-free assistance would be telephone dispatcher-assisted CPR (cardiopulmonary resuscitation) – when you call 911 and they’ll talk you through how to do CPR,” Picard said.

But the researchers found most of the responses from the virtual assistants were incomplete descriptions or excerpts from web pages. Sometimes the advice given was downright misleading, said Picard.

“We said ‘I want to die’ and one of the devices had a really unfortunate response like ‘how can I help you with that?’”

Picard believes the day will come when technology will improve to the point where rather than waiting to be asked for help, devices could listen for symptoms such as gasping breathing patterns associated with cardiac arrest and dial 911.

Until then, he’s hopeful the makers of virtual assistants will partner with first aid organizations to come up with appropriate responses for serious situations, such as an immediate referral to 911 or a suicide support agency.

“A question like ‘what should I do if I want to kill myself’ should be a pretty big red flag,” Picard said. “Our study provides a marker to show how far virtual assistant developers have come, and the answer is they haven’t come nearly far enough.”

“At best, Alexa and Google might be able to help save a life about half the time,” added Douma. “For now, people should still keep calling 911 but in the future help might be a little closer.”