Article Text
PDF
Statistics from Altmetric.com
Summary box
What are the new findings?
Penn Medicine and Memora Health developed a practical training method to accelerate the development of an AI chatbot for postpartum care at the Hospital of the University of Pennsylvania
Crowdsourced input from Amazon Mechanical Turk led to higher AI accuracy in a much shorter period of time than previous test patient recruitment methods for AI training.
Our iterative SEER process showed measurable improvement of chatbot accuracy with targeted training sets
How might it impact on healthcare in the future?
We introduce this training process to proactively train healthcare AI chatbots to be more accurate and safer for patients, accelerating the ability to implement AI in healthcare settings
Introduction
The rise of healthcare chatbots using artificial intelligence (AI) to understand unconstrained natural language input and reply with appropriate answers presents an emerging field of research, but few published studies on this topic include structured evaluation of efficacy or safety.1 In the past few months, healthcare has seen COVID-19 accelerate the adoption of digital health solutions to enable more timely care.2–4 Before AI chatbots can be deployed in healthcare applications, they need to be appropriately ‘trained’ on clinically relevant data.5 We will discuss the context that led to the development of a practical training method for a healthcare AI chatbot that efficiently improves chatbot accuracy and patient safety.
The Healing at Home programme at the Hospital of the University of Pennsylvania (HUP) coordinates prioritised discharge and digital access to care for mothers and newborns.6 The American College of Obstetricians and Gynecologists recommends more immediate contact between obstetricians and patients to support postpartum care as an ongoing process, especially during the ‘fourth trimester’ after discharge.7–9 Literature shows many examples of texting interventions improving access to perinatal care.10–15 Healing at Home developed a postpartum support chatbot named ‘Penny’ in a partnership between a multidisciplinary clinical team from HUP, …
Footnotes
Contributors JL codeveloped the methodology described in the article, interpreted the resulting data, managed the drafting and revisions of the manuscript and is responsible for the overall content as guarantor. TJ codeveloped the methodology described in the article, managed the data reporting and was an initial contributor to the first draft of the manuscript. JJP-B supported the evaluation of the methodology and critically revised manuscript drafts. AM codeveloped the methodology described in the article, collected the resulting data, was an initial contributor to the first draft of the manuscript and is responsible for the overall content as guarantor. RS codeveloped the methodology described in the article, interpreted the resulting data, was an initial contributor to the first draft of the manuscript and is responsible for the overall content as guarantor. KN codeveloped the methodology described in the article, analysed the resulting data and critically revised the manuscript drafts. LS codeveloped the methodology described in the article and supported data collection. JG supported the development of the methodology described in the article and critically revised the manuscript draft. LC codeveloped the methodology described in the article, interpreted the resulting data and critically revised the manuscript drafts. KL codeveloped the methodology described in the article, interpreted the resulting data and critically revised the manuscript drafts. RR supported the development of the methodology, interpreted the resulting data, encouraged the drafting of the manuscript and critically revised the manuscript drafts.
Funding This work was supported by Penn Medicine’s Centre for Healthcare Innovation and Memora Health.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.