Statistics from Altmetric.com
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
What are the new findings?
A simple web-based mobile application to estimate risk of COVID-19 transmission is feasible and acceptable among the US public.
Transmission risk can be estimated for app users based on local prevalence of disease, type of activity and mitigation measures employed, without collecting personal health information.
How might it impact on healthcare in the future?
Health apps that are free, publicly available, and incorporate evidence-based research could reduce COVID-19 fatigue and safety measure compliance by allowing individuals to make their own risk assessments and enjoy low-risk activities safely.
Social media may be a useful tool to obtain early user feedback and promote health tools during a public health emergency.
The transmission dynamics of SARS-CoV-2 are complex. They depend on factors that enhance or protect against fomite, large droplet and aerosol transmission, as well as local prevalence of disease. The public face challenges in understanding and making educated decisions about daily activities, prompting perspective pieces such as ‘We’ve been left to calculate our virus risk on our own. We’re terrible at it.’1 Mobile apps could play an important role in helping individuals understand infection risk from everyday activities. Current COVID-19 risk apps range from predictive models estimating the risk of critical illness, to symptom checkers and workplace guides.
Here we present the development and implementation of MyCOVIDRisk app, intended to both inform Americans of the risk incurred when engaging in different activities and to guide on risk-reduction measures. Our objective was to create a tool that was freely accessible to the public, incorporated up-to-date information on local disease prevalence2 and helped people easily understand how to reduce risk without divulging personal information. The hypothesis was that if individuals could continue to engage in enjoyable low-risk activities, we could reduce community transmission while also minimising anxiety, isolation and so-called pandemic fatigue.
The idea of MyCOVIDRisk was conceived in July 2020 following conversations on social media about challenges with estimating risk and the cognitive burden of making these calculations with little knowledge several times a day. We reviewed peer-reviewed and grey literature to identify published studies on: (1) transmission dynamics and protective measures, (2) COVID-19 risk scores, and (3) risk assessment apps or websites. We aimed to identify infection (1) risk factors, (2) sources of reliable prevalence data, (3) attack rates associated with different activities and (4) studies modelling the effect of mitigation factors.
In contrast to explanatory statistical modelling that focuses on testing hypotheses, our goal was to create a predictive model for the purpose of forecasting the value of a new observation (whether a person will develop COVID-19). We aimed to identify a simple model that would roughly predict infection risk and also make the development process shorter and less complicated. With this knowledge in mind, we undertook four steps common to predictive model development: data understanding, model assembly, model audit and model delivery. We had an a priori understanding of variables that needed to be included to predict transmission risk based on existing literature, clinical care and public health guidelines, and supplemented this experiential data with a literature review (data understanding). Based on the literature review, we identified high-quality models for estimating transmission and mitigation (model assembly). We consulted with experts in biostatistics, epidemiology and mathematics to inform adjustments to the model and to provide independent assessments model validity (model audit). Due to continued lack of accurate population data on transmission patterns, prospective or retrospective model validation based on real-world data was not possible at the time of model creation. After we completed fine tuning of the model and received feedback on the app design, we deployed the app and shared it publicly together with documentation and communication of the scientific premise of the model (model delivery).
Creation of app wireframe
Design decisions were made based on behaviour change theory, theories of ‘persuasive technology,’ principles of user-centred design, and prior experience in development of effective and engaging digital health technologies, to ensure the app was usable for individuals of all ages and digital literacy.3–5 To maximise persuasiveness, we designed the app to be used in two stages: the first stage requires user input to specify details of the planned activity, and the second stage allows the user to choose options to reduce transmission risk (online supplemental file 1). To reduce user fatigue and improve engagement, we limited scrolling, avoidable clicking and the number of input screens prior to the preliminary output. Consistent with best practices for digital health behaviour change, we provided personalisation and interactivity, used multiple techniques for engagement, and included both positive and negative feedback. We worked with a UX expert to design icons that were visually appealing and inclusive.
We obtained basic usage statistics from Google Analytics (14 October–18 December 2020) and back-end app data (1 October–18 December 2020). Although formal user feedback was not solicited after launch, unsolicited feedback was received through our website, email and informal conversations.
Findings of our review included a risk chart ranking day-to-day activities into categories of risk and COVID-19 risk apps6; however, the apps required users to share demographic information, chronic health conditions or health records.7–10 Many COVID-19 apps were designed to show risk of critical, fatal illness or hospitalisation. Other than the transmission estimator by Jimenez, we did not find other COVID-19 tools that calculated projected risk of daily activities.11
Based on our initial review, risk factors included location near high COVID-19 prevalence counties, indoor activities,12 poor ventilation, long durations of visits, physical exertion and close proximity to others.13 Mitigation factors included wearing a mask, distancing, reducing activity time, washing hands, increasing ventilation and wearing eye protection.14–19
Based on our literature review, the most accurate model of transmission dynamics was identified as the box model of airborne transmission, developed by Miller et al and instrumentalised in the COVID-19 Aerosol Transmission Estimator by Jose Jimenez.11 20 Using Jimenez’s estimator, we calculated the probability of infection given user entered data, local prevalence, and then used odds ratios (ORs) reported in the literature to calculate posterior probabilities of infection with mitigation measure use.16 After consultation with external experts, in the absence of a clear consensus of how to calculate risk, we assumed that the individual protective measures were independent events with independent effects on probability (eg, allowing multiplication of effects). Regarding risk levels, we considered a 5% risk of infection (eg, the attack rate for a family member) as ‘very high,’21 and the risk of fatality when flying in an aeroplane (assuming travelling by plane eight times a year during a 75-year lifespan) as ‘very low’.22 Parameters for other user inputs—quanta (infectious particle transmission rate), building ventilation rates, event venue size—were sourced from peer-reviewed literature and expert consensus.11 23
Based on iterative user feedback on the initial app wireframe, we reordered screens; redesigned input icons; changed input screens for activity time and per cent of people masked; included check-boxes to ensure its use in the USA and moved detailed modelling information to optional screens.
Analytics on use
MyCOVIDRisk app was launched on 1 October 2020. As of 18 December 2020, the app was accessed over 1 million times by users in over 112 countries (96.5% in the USA). Within the USA, first-time users accounted for 84.5% of access (table 1). Of activities selected, meeting at friend’s house was most common (22.6% of respondents), followed by shopping (20.2%) and taking a walk (11.6%). Activities related to dining (restaurant: 8.7%, bar: 1.7%) and self-care (salon: 7.0%, gym: 5.7%) were less common (online supplemental file 2). Planned gathering sizes varied widely between users with the majority of calculations (55.6%) involving groups of 1–10 people. Of those using mitigation steps, almost all (99.9%) selected social distancing and 83.7% planned to wear a mask (table 2).
Tracking user risk assessments before and after selection of mitigation steps revealed that the majority of users received a ‘low-risk’ or ‘very low-risk’ assessment even before mitigation steps were selected. Those that received a ‘high-risk’ score most often were able to achieve ‘low risk’ after selecting mitigation steps (online supplemental file 3).
The MyCOVIDRisk app was created within 3 months in response to the public health imperative for accurate, comprehensible risk assessment information. Its high utilisation, despite lack of formal advertising, demonstrates demand for and accessibility of this simple risk assessment and mitigation tool.
Using health apps to increase public health awareness and reduce misinformation should be part of a comprehensive public health strategy to address epidemics or pandemics. Over 81% of adult Americans have smartphones and one in five uses health apps.24 25 To design and launch a useful, usable application requires not just scientific evidence, but also the ability to incorporate principles of user-centred design and science communication. Behaviour change is essential to reducing SARS-CoV-2 transmission. Elements of behaviour change related to this work include: (a) helping people understand transmission (here is your MyCOVIDRisk score), (b) creating social norms (people want to reduce risk), (c) giving people an action (take these mitigation steps like mask-wearing to reduce risk), (d) making change easy (easily choose a safer activity). MyCOVIDRisk is easy to use, has widespread uptake and illustrates the importance of multiple layers of protection. Additionally, it is updated with real-time prevalence data using Application Programming Interfaces (APIs) and is less expensive than other traditional public messaging campaigns. Using health apps to increase public health awareness and reduce misinformation should be part of a comprehensive strategy to address pandemics.
Future work should include considerations of how to disseminate and motivate use of the app by those who may be sceptical or unaware, and how to enhance use of mitigation steps. Changes in knowledge, behavioural intention and actual behaviours are still unknown. Additional modifications could include enhancement of more complex risk modelling (eg, travel, doctor’s visits), ‘behavioural nudges’ or linkages to testing. Limitations include that it may be inaccessible to those at highest risk: Black, Hispanic, Native Americans and older adults have decreased access to broadband WIFI (although national studies suggest similar rates of smartphone access and health app usage). We hope to translate the app and ensure cultural relevance to diverse groups.
Although the risk model would ideally be validated prospectively, continued lack of accurate data on exposure histories of those diagnosed with COVID-19 makes this challenging. Effect estimates of mitigation measures were partly based on observational studies of other beta-coronaviruses, due to limited data available for SARS-CoV-2. We purposefully provided quintiles of risk rather than exact estimates, recognising continued scientific debate about precise transmission dynamics. Although we may be overestimating the benefit of multiple protective measures, research shows that when layers of protection are used the risk approaches zero.26–29 We would encourage scientists to contact us to pressure-test our model using assumptions about viral transmission dynamics.
MyCOVIDRisk could serve as a model of mobile apps that enhance public awareness and gamify risk mitigation. Although the impact of the app on COVID-19 fatigue and anxiety has not yet been elucidated, apps such as MyCOVIDRisk may help the public make more nuanced decisions that allow safe activities to continue when pandemics last for months.
The authors would like to give thanks to other scientists who have given their input and feedback during the development of this app: Jose L Jimenez (University of Colorado-Boulder) and his work on the COVID-19 Aerosol Transmission Estimator, Roee Gutman (Brown University), David Blake (Augusta University), Arnold Barnett (MIT), Raina MacIntyre (University of New South Wales), Thomas Tsai (Brigham and Women’s) and the Harvard Global Health Institute.
EMG and CSB are joint first authors.
EMG and CSB contributed equally.
Contributors MLR and EMG, researchers and emergency medicine physicians associated with Brown University, are both co-creators of the MyCOVIDRisk app. They both equally shared in the ideation, development and dissemination of this innovative digital health tool, and assisted in writing this paper. EMG and MLR co-lead the creation and refinement of the risk assessment calculator. CSB, a senior research assistant at the Brown-Lifespan Center for Digital Health as well as the project coordinator for the MyCOVIDRisk app, assisted in the design, development and dissemination of the tool, co-led the writing of the paper and assisted with analysis. SP, a medical student at the Warren Alpert Medical School of Brown University, performed the data analytics and assisted in writing. All authors contributed intellectually to the content of the paper and reviewed and edited the final manuscript.
Funding This publication was made possible by the National Institute on Aging (K76AG059983), the Brown Physicians Academic Assessment Research Award (PI: EMG), the Warren Alpert Endowed Professorship of Emergency Medicine (MLR) and the Brown-Lifespan Center for Digital Health.
Competing interests EMG reports grants from the National Institute on Aging (K76AG059983) and the Brown Physicians Academic Assessment Research Award (PI: EMG) during the conduct of the study. MLR reports funding from the Warren Alpert Endowed Professorship of Emergency Medicine and the Brown-Lifespan Center for Digital Health during the conduct of the study.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.