Aim The technology of online and mobile-based symptom checkers is developing on a background of unprecedented demand for General Practioner (GP) and medical services. This study was performed to assess the performance of popular ‘artificial intelligence’ symptom checkers currently available to the public and evaluate their accuracy as a screening tool for head and neck cancer symptoms.
Method We selected three of the most popular and previously compared mobile application (app) symptom checkers available including Babylon (London), Your.md (London) and Ada (Ada Health GmbH, Munich). A recent Pan-London Suspected Cancer Referral Guide was obtained and distilled into a list of qualifying symptoms for referral to secondary care on a 2-week wait cancer pathway. A generic patient background was devised and each symptom was passed through the selected symptom checker apps. Differential diagnoses and recommended triage times were compared.
Results Overall a cancer diagnosis was suggested possible in the differential diagnosis in 33% of symptoms. As a screening tool, Babylon was the most accurate for including a potential cancer diagnosis in the differential diagnosis, with a sensitivity of 45% cases, Ada with 32% and Your.md 23%. Ada attempted the most diagnoses, able to provide a differential diagnosis in 95% scenarios, but was also the most risk averse triage assessor, with 55% outcomes recommending medical advice immediately or within hours.
Conclusion The ability of modern symptom checkers to promote health awareness and self-checks is promising, but we have identified a number of areas of potential improvement. At present the sensitivity of the tested symptom checker apps remains low for head and neck cancer.
- head and neck cancer
- medical apps
- symptom assessment
- medical informatics
Statistics from Altmetric.com
Contributors Conception and design, acquisition of data, analysis and interpretation of data, writing of manuscript: JF. Analysis and interpretation of data, writing of manuscript: JPJ.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement All data relevant to the study are included in the article or uploaded as supplementary information.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.