Article Text
PDF
Abstract
Background Numerous groups have developed software applications (apps) for use by community health workers (CHWs) in hard-to-reach settings. However, these have either not been based on clinical guidelines or are not freely available. Our objectives were to (1) design and develop an app for use by CHWs; and (2) conduct preliminary testing of the app to identify potential obstacles.
Methods We used the principle of human-centred design to develop an app programmed to an Android operating system. We used a mixed-methods approach which included site observations, meetings with stakeholders and the app development team, and laboratory simulation to fine tune the design. The ‘Mobile Application Rating Scale’ (MARS) was used for testing reliability and quality. We also assessed the validity of the app by matching the uploaded data with ‘gold standard’ preset answers.
Results Depending on human–computer interactions, the app has reminder, advisor, critic and guide functions which can facilitate CHWs to make clinical decisions. We found the app is usable based on the final score of the MARS tool, and that the entered data were accurate. We present the simple procedures that were followed to develop this Android app. The app, including all of its code, is freely available.
Conclusion The app shows promise as a tool for the management of non-communicable disease in a rural setting in India. The next step will be to refine the app in a field setting and then to evaluate its efficacy in a large-scale clinical trial.
- public health
- cardiovascular diseases
- diabetes mellitus
- early diagnosis
- health services research
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
WHAT ARE THE NEW FINDINGS
A major barrier to providing healthcare is the lack of tools for community health workers (CHWs) in a rural setting.
We describe the design of an app for clinical decision support of CHWs for management of non-communicable diseases.
CHWs were able to use the app to screen and manage people with non-communicable diseases without detailed knowledge of guidelines or treatment protocols.
HOW MIGHT IT IMPACT ON HEALTHCARE IN THE FUTURE
The app provides support to overcome barriers in settings with limited health infrastructure by facilitating screening, referral and electronic data capture.
The app, including all of its code, is freely available.
Introduction
The burden of non-communicable diseases (NCDs) such as cardiovascular disease, diabetes, lung disease and cancer has been increasing globally, including in India.1 2 People living in India or other low/middle-income countries (LMICs) have approximately twice the risk of dying from NCDs than people from high-income countries.1 Barriers to early detection and management of NCDs in rural India include lack of awareness, unaffordability and inaccessibility of healthcare, a shortage of doctors and doctor absenteeism in public health centres.3 4 These barriers can be at least partially overcome by task sharing with community health workers (CHWs). For example, a CHW-led group-based education and monitoring intervention was found to be effective in controlling hypertension in rural India.5 However, CHWs require support to be able to diagnose and manage patients with NCDs, which could be provided through an app-based clinical support system.
Clinical decision support technology, to support CHWs in rural areas, could potentially be applied using handheld tablet devices. Such approaches have been used to improve drug-prescribing practices,6 7 reduce serious medication errors,8 enhance delivery of preventive care services9 and improve adherence to recommended standard care practices.10 11 Thus, numerous groups have developed mobile health (mHealth) interventions or software applications (apps) for use by healthcare workers in hard-to-reach settings.12 However, these have either not been designed for CHWs, not followed relevant national guidelines or not been freely available.13–18 Moreover, we are not aware of any available descriptions of the pathway to development of a decision support system for CHWs in an LMIC setting. Indeed, as identified in a recent systematic review, there has been a conspicuous absence of science-based methods and theory-based frameworks for designing and developing mHealth interventions in LMICs, due in part to the presence of setting-dependent factors which have limited progress.19
Use of mHealth has become familiar in India and other LMICs to support CHWs in their job.20 To be able to use mHealth, CHWs require basic knowledge of how these devices operate so they can be trained in the use of the specific app. We developed an app for CHWs which can be installed on a tablet device. These devices are then given to selected CHWs and detailed training is provided on the use of the tablet device and the app.
Herein, we describe the development of the ‘Arogya Sahyog’ app (a Hindi term meaning ‘health assistant’), a modern clinical decision support system. We used a human-centred design (HCD) approach to develop the app.21 This is a process for gaining insight into the needs of the beneficiaries of our innovation, creating sound engineering approaches to meet their needs and delivering solutions that work in specific contexts.21 The objectives of this study were to design and develop the software application, incorporating a clinical algorithm on NCDs, to be used by CHWs; and conduct preliminary testing of the stable version of the app to identify potential obstacles. Adequate budget allocation is crucial for successful development of an app. Therefore, we also included an estimate of the total financial cost of the app development. The app, including all of its code, is freely available (https://figshare.com/s/1067ac251be4a2fa79d6).
Methods
Premise and design
We used the approach of the Innovation Design Engineering Organisation to HCD, which involves three stages: hear, create and deliver.22 Both in-house and field testing were undertaken to prepare the various components of the decision-support tool. To meet the research objectives of each step, we used a mixed-methods approach which included site observations, meetings with stakeholders and the app development team, laboratory simulation, and in-context reliability and quality testing (figure 1).
Flow chart showing the various stages of app development UX (user experience), UI (user interface) and QA (quality assurance). QA testing of the initial version includes functional testing, performance testing and security testing.
Existing documents and literature review
We conducted a scoping review to identify available information and communication technology (ICT) interventions used in NCDs, which are also suitable for older adults.23 This also gave us insights into the barriers to, and enablers of ICT, including use of clinical decision support systems. We also explored various guidelines and training modules for Accredited Social Health Activists (ASHAs, a cadre of female CHWs in rural India) related to the detection and management of NCDs in India,24–26 so that this information could be incorporated into the app. ASHAs, who are locally recruited by the state government, have a minimum level of education of 10 standard grades. ASHAs may also have some basic computer literacy. However, this is not a mandatory requirement. They are eligible to receive training to acquire the necessary knowledge, skills and confidence to perform their designated roles in relation to NCDs. Thus, we developed training materials and arranged training sessions to facilitate their competency in the use of the Arogya Sahyog app.
Site observation and consultation with frontline health workers
A field team conducted a structured observation of the routine practice of ASHAs in rural India. We purposefully selected four villages located at Chamba, in the mountainous Tehri District of Uttarakhand in India, where accessibility to health services is often difficult for residents. Depending on the size of the population, each village is usually covered by one or two ASHAs. ASHAs’ roles are to register all villagers aged at least 30 years; complete a Community Based Assessment Checklist, including risk assessment; and undertake health promotion activities for NCDs.24 Each site was observed for 3–4 hours by a trained ethnographic researcher. Data from the site observation were in the form of field notes. The purpose of this process was to determine preferences and choices for the features of the app, thus allowing derivation of the data flow structure (online supplemental figure 1).
Supplemental material
Stakeholder consultation to set the requirements for the decision support system
We consulted members of the app development team, which consisted of clinicians (RS, AS), epidemiologists (RajS, PS), public health experts (SBZ, RGE and AGT) and computer engineers (TF, NDS) to identify the requirements for the hardware and software of the clinical decision support system (online supplemental table 1). Use of a traffic light system (red, yellow, green) in the app was recommended, so that ASHAs could easily identify and refer people with NCDs who need immediate medical attention. The investigators used a checklist,24 standard laboratory assessments and an existing validated questionnaire27 to ensure the adequacy of data collection (online supplemental table 2).
Laboratory simulation testing
Using a set of 25 mock patients whose clinical histories were based on actual clinical conditions (pseudo-patients), the app development team ran the algorithms of the clinical decision support system to assess the accuracy of the system. A checklist was used to ensure that the message content required for specific clinical conditions worked properly. During this iterative process, any time the decision support system produced an inaccurate message or failed to generate an appropriate message, the system rules were refined. The laboratory simulation process was repeated until the team received a clinically appropriate decision for each patient. Quantitative reports of the number of false negatives, false positives and correct messages during each iteration of laboratory testing were internally reviewed.
User experiences
Five dummy participants took part in the pretesting phase of the alpha version of the software. All participants were found to be technically sound in operating tablet devices and computers (online supplemental table 3). After brief training on the app, they performed a functionality test using a Galaxy Tab A (Samsung, Korea) tablet device to install and run the app. Their feedback was used to fix bugs and refine the app.
In-context reliability and validity testing
We developed four case scenarios in hard copy with ‘preset answers’ to test the reliability of the app. Four members of the research team, who were either supervisors or facilitators of the ASHAs, were included to form the App Testing Field Team. These four team members, who were computer literate (online supplemental table 3), were briefly trained on how to use the app before they entered data. Later, we downloaded their data entries from the server and matched those with the hard copies of preset answers. Then, we calculated the average error rates to report the fidelity of the app. We purposively used three types of tablet devices: Galaxy Tab A (Samsung, Korea), Snap 4G2 (iBall, India) and RPTPE0801 (Reconnect, India) to test the beta version of the app. This part of the test was performed at the field site in India (Chamba, Tehri, Uttarakhand). Finally, the ‘Mobile Application Rating Scale (MARS)’ was used to evaluate the quality of the app.28 29
Creation of a patient profile in the app and a dashboard web application
We also developed and tested a simple web-based dashboard to enable facility managers and supervisors to view analyses of the app’s data, trends, summaries, etc (online supplemental figure 7). All codes for the dashboard are freely available (https://figshare.com/s/b4d49f3f5ec124996e6b).
Cost analysis
We estimated the development cost and time for the staff. Three staff members, one a technologically skilled researcher (SBZ) and two who are experienced app developers (NDS and TYG), were asked to provide the number of hours they spent to complete a set of minimum viable app features. From the survey data, we conducted a simple calculation of the time and nominal hourly charges ($A60) to estimate the total development costs of the Arogya Sahyog app.
Data analysis
We used the MARS tool to assess the app performance.29 The MARS tool contains 23 items of questions divided into five sections: engagement, functionality, aesthetics, information quality and subjective quality. Each item is scored using a 5-point Likert scale (1: inadequate, 2: poor, 3: acceptable, 4: good and 5: excellent) and a mean score is given for each section. Finally, the mean values of the first four sections were used to provide an absolute measurement of the app quality. We used Microsoft Excel (Microsoft Windows, USA) to analyse data. Simple descriptive statistics were used to present the findings.
Results
Hear
Researchers conducted literature reviews, site observation and meeting sessions, and conducted interviews to assess the requirement of the app among the ASHAs and understand the sociocultural context of technology use during the hear phase. Most of the ASHAs we interviewed were accustomed to operating smartphone devices. We heard that the ASHAs desired software that can facilitate their ability to make clinical decisions. More specifically, they desired software with reminder, advisor, guide and visual display functions (online supplemental table 4). Details of specific findings and their implications for the various methodological approaches are presented in table 1.
Findings and implications of various methodological approaches
Create
During the create stage, as the team developed and refined the clinical decision support system to integrate care of patients with or at risk of NCDs, findings from laboratory simulations and in-context usability testing were immediately translated into prototypes. The decision support system was developed into two stages: (a) development of a clinical algorithm and (b) development of the Arogya Sahyog app.
Development of the clinical algorithm for the app
We prepared and incorporated the clinical algorithm in the app to generate a provisional diagnosis (eg, hypertension, pre-diabetes or diabetes, hyperlipidaemia and anaemia) and specific instructions based on data collected from the patients (online supplemental figures 2 and 3). Hence, computer-based algorithms in the Arogya Sahyog app produce and display messages that are patient specific, educational and designed to inspire behaviour change by promoting healthy lifestyles (online supplemental figure 4).
Development of the app ‘prototype’ and app enhancement
The prototype software was developed on 21 July 2019 and installed on a tablet device (online supplemental figure 5). The app allows interoperability features (synchronised with other systems) and can be linked with any database. The initial pre-alpha and alpha (15 August 2019) version was released and tested by the researchers. Subsequently, researchers (SBZ, RGE and AGT) from our institute determined whether all modifications listed in the set of criteria, including the traffic light system, had been adequately implemented. If a necessary action is not completed, most commonly due to delays in collecting clinical data, the reminder is repeated in the dashboard of the app during the next patient visit (online supplemental figure 6A). An inbuilt dashboard and a patient profile were developed (online supplemental figure 6B,C) in the app to help CHWs monitor and follow up the enrolled patients. The system was tested adequately to check whether it can produce individualised and tailored reminders and display this information on the screen regardless of access to internet networks. Once the researchers were satisfied with the app, this alpha version was approved for pretesting. Finally, after software testing and incorporating feedback from users, the stable alpha version was ready for field testing in India.
Validity and reliability testing of the app
A written user manual was developed illustrating the structure and functions of the Arogya Sahyog app (https://figshare.com/s/a2df83791dbbd9998aac). The App Testing Field Team used the Arogya Sahyog app to enter the data for the case scenarios. After assessment, we found the error rate ranged from 0.0% to 0.3% (online supplemental table 5). The final score of the MARS tool was 4.4 out of 5, with the App Testing Field Team rating various ranges for engagement (3.8–4.6), functionality (3.8–4.8), aesthetics (4.0–4.8), information quality (4.0–4.8) and subjective quality (4.2–4.6) (figure 2).
Scoring of the Arogya Sahyog app by the App Testing Field Team according to the Mobile Application Rating Scale (MARS). Scoring of five sections of MARS was conducted by four members (P1, P2, P3, P4) of the App Testing Field Team. Average values of the first four sections are used to provide an absolute ‘mean score’ of MARS for the app.
Cost analysis
Making a simple app such as Arogya Sahyog took about 350 hours of investment. We estimated the total cost for developing the app was approximately $A21 000. For this project, we also developed a dashboard web application, which incurred additional 40 hours (~$A2400) of investment. The specific cost breakdown for each component of app and development of the dashboard web application is provided in online supplemental table 6.
Deliver
Findings from the laboratory simulations, validity and quality testing were used to refine the clinical decision support system. Finally, the beta version of the app was released on 1 November 2019 (https://figshare.com/s/ae45d074efb4d7945b62). Then, the investigator team assessed potential challenges to implementation of the app in the field and commenced a feasibility study to assess usability and acceptability of the app among ASHAs. Primary activities included: purchasing new tablets, sim cards, internet package and pocket Wi-Fi routers, and hiring and training staff to collect data. We found that the new app was easy to install and use with all three brands of tablets that we used in the laboratory.
Discussion
We used HCD principles and practices to understand the opportunities for integrating mHealth into the management of risk factors for NCDs in a resource-poor setting, and to improve case detection, initial management and referral. During the hear stage, the team gathered data to inform the development of the Arogya Sahyog app through observation sessions, stakeholder consultation and interviews. The beta version of the app appears to have excellent quality as assessed through the MARS score.
The Arogya Sahyog app is a clinical decision support system for early detection and initial management of people with NCDs, enabling task sharing between medical professionals and ASHAs. This app is programmed to an Android operating system, as these tablet devices are widely available and affordable in LMICs, including India. The app worked well on all three brands of tablets we tested. It also appeared to be easy for the App Testing Field Team, with minimal training, to use, since fidelity against ‘preset’ answers was high.
A major limitation of many mHealth interventions is the need for continuous network access. Indeed, we found that internet signal strength in some rural areas around Chamba was poor. The Arogya Sahyog app overcomes this obstacle because it was designed to work mostly offline, with data being uploaded only when the internet is available (eg, weekly). Thus, availability of good network coverage within villages is not necessary for use of the Arogya Sahyog.
The HCD approach to the design and development of the Arogya Sahyog app facilitated input from critical stakeholders, the CHWs themselves. We have made the codes of the app freely available (https://figshare.com/s/1067ac251be4a2fa79d6) for the benefit of young researchers and app developers. The app is very context dependent, so may require some level of modification for use in other settings. Nevertheless, the availability of codes and our description of the app herein should provide a useful guide for those who want to develop similar mHealth apps for LMIC settings.
Limitations
Our preliminary testing of the app provided valuable insights into the first phase of the app development. However, our methodological approach is not without limitations. First, our reliance on purposeful sampling strategies could limit the generalisability of the findings. Second, given that members of the research team assessed the app, reporting bias may have occurred. However, the impact of this potential bias was mitigated by use of multiple methods to triangulate findings. Third, the number of participants in the pretesting and validity testing was small. However, their responses were very consistent, indicating that this small sample size was adequate for our purpose of generating advanced feedback to improve the functions of the app. Fourth, the participants who were involved in the initial testing of the app had sound knowledge regarding the basic aspects of computer operation and software use. However, this may not be the case for the majority of CHWs in the local context. Therefore, the next step in this project is to provide selected ASHAs with extensive training on the use of the Arogya Sahyog app and to evaluate its effectiveness. The protocol for this feasibility study has been described in detail previously.30 Thus, despite the mentioned limitations, the current study provides the first step in the pathway of development of the Arogya Sahyog app, which should be considered its main strength.
Conclusions
The Arogya Sahyog app shows promise as a tool for managing NCDs in a rural setting in India. The next step is to validate and refine the app in a field setting among ASHAs and then to evaluate its efficacy in a large-scale clinical trial. This trial has been registered in the Australian and New Zealand Clinical Trials Registry (ACTRN12620000436976).31
Ethics statements
Patient consent for publication
Ethics approval
This study involves human participants and was approved by the Ethics Committee of the Garhwal Community Development and Welfare Society (GCDWS/AROGYA/ETHICAL) and the Monash University Human Research Ethics Committee (#23168). All participating ASHAs and participants signed an informed consent form. Permission was taken to conduct the observation of study sites and direct observation of ASHAs from their supervisors.
Acknowledgments
We express our gratitude to the ASHAs who participated in this study. In addition, the authors are grateful to Farid Raisi, Zubair Ahmed, Hem Prakash, Manoj Kumar, Mulugeta Birhanu, Lana Coleman and Rajshree Thapa for their support in testing the app and conducting the training.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors SBZ, RGE and AGT were responsible for the initial draft of the manuscript. SBZ, RGE, RajkumariS, RajeshS, PS, AS, TYG, NDS and AGT provided input on subsequent drafts of the manuscript and were responsible for the design, development and testing of the Arogya Sahyog app.
Funding This work is supported by intramural funding from Monash University, Australia. SBZ received a scholarship from the Australian Government Research Training Program (RTP) in support of his academic career.
Disclaimer The funding source had no role in the design, implementation, analyses, interpretation of the data or decision to submit results.
Competing interests RGE reports grants from the National Health & Medical Research Council (Australia) outside the submitted work; and has received consulting fees from Medtronic Australasia, in relation to work other than that described in this manuscript. AGT reports grants from Monash University, during the conduct of the study; and grants from the National Health & Medical Research Council (Australia), outside the submitted work.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.