Background Virtual digital assistants are devices that interact with the user through natural language processing and artificial intelligence. They can respond to verbal requests for first aid information. This study analyses the responses provided by the four most common devices.
Methods This mixed-methods study employs structured interviews of the virtual digital assistants (Alexa, Cortana, Google Home and Siri) as well as descriptive statistical analyses. One hundred and twenty-three interview questions, based on 39 first aid topics, were employed. Responses were analysed for recognition and quality. Detection of query acuity was performed according to triage guidelines and response complexity was calculated.
Results Device performance was highly variable. Alexa and Google Home demonstrated high rates of recognition (92% vs 98% (p=0.03)) and low-to-moderate congruence with guidelines (19% vs 56% (p=0.04)). They appropriately recommended emergency response system activation 46% of the time vs 16% (p=0.01) of the time, respectively. The overall low quality responses of Cortana and Siri prohibited their analysis. Mean response complexity for Alexa was ‘grade 10’ vs ‘grade 8’ for Google Home (p<0.001).
Interpretation This is the first study to assess virtual digital assistants from a first aid and basic life support perspective, finding potential in this technology to provide life-saving instructions and activate the emergency response system. When asked common first aid related questions Google Home and Alexa outperformed Siri and Cortana. Overall, the device responses were of mixed quality ranging from the provision of factual guideline-based information to no response at all.
- medical apps
Statistics from Altmetric.com
Twitter @CtPicard, @matthewjdouma
Collaborators Dr Alison Clifford and Dr Shubha De.
Contributors CP and MJD planned and coordinated the study. CP, KES, KP and MJD participated in the pilot. CP and MJD performed data collection and graded congruence of recommendations with KES as a tiebreaker against guidelines. KES and MJD compared device ability to recognise acuity to pCTAS. Complexity grading was performed by CP and MJD. Statistical analysis was performed by MJD. All authors contributed to manuscript preparation. MJD submitted the study, and revisions were performed by CP and MJD.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient consent for publication Obtained.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement Data are available in a public, open access repository. We are storing the data on our website www.RescueScience.org. It includes all the documents related to this study including VDA interview transcripts and excel spreadsheets and stats analyses.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.