Article Text

other Versions

PDF

Original article
Perceptions of data processes in mobile-based versus paper-based health information systems for maternal, newborn and child health: a qualitative study in Andhra Pradesh, India
  1. Lillian H Nguyen1,
  2. Amnesty E LeFevre2,
  3. Larissa Jennings2,
  4. Smisha Agarwal2,
  5. Garrett Mehl3,
  6. Alain B Labrique2,
  7. Lakshmi Durga Chava4
  1. 1Department of Health, Behavior, and Society, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland, USA
  2. 2Department of International Health, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland, USA
  3. 3Department of Reproductive Health and Research, World Health Organization, Geneva, Switzerland
  4. 4Human Development Unit, Society for Elimination of Rural Poverty, Hyderabad, India
  1. Correspondence to Dr Amnesty E LeFevre, Department of International Health, Johns Hopkins Bloomberg School of Public Health, 615 N. Wolfe Street Baltimore, MD 21205, USA; aelefevre{at}gmail.com

Abstract

Background Many maternal, newborn and child health (MNCH) programmes have paired community health workers with mobile technologies to strengthen the ability of health information systems (HIS) to track women and children across time and beyond the clinical setting. However, little is known regarding the comparative effectiveness of using mobile technologies to enhance HIS data in resource-poor settings.

Methods Focus group discussions were conducted with community health workers called Health Activists (HAs; n=30), Community Organisation Leaders (n=28), HA Trainers (n=21), district and tribal area officials (n=3) and State Officials (n=4). We analysed user perceptions along seven key HIS processes: data collection, transmission, processing, analysis, display, quality checking and feedback.

Results The mobile-based health information system (mHIS) was found to be supportive of the MNCH continuum of care by improving the regularity and timeliness of access to robust data. Respondents noted that data errors were reduced in real time through automated error checking and data processing, which also reduced users’ workloads. The mHIS additionally enabled users to analyse both individual and aggregate data, allowing them to identify specific individuals in need of services or training as well as to identify general trends in service delivery. The system's data display and feedback mechanisms were viewed as improving data use for decision-making. The remaining challenges of the mHIS versus the paper-based HIS included resource, infrastructural and technological barriers that hindered efficient use over time.

Conclusions As compared to paper-based HIS systems, mobile technologies can improve health information processes in resource-poor settings. More efforts are needed to ensure sufficient financial investment, training and use of mHIS data at all levels of the HIS.

  • Accessible
  • Global Health
  • mHealth
  • Nutrition
View Full Text

Statistics from Altmetric.com

Introduction

Sound data on the delivery of maternal, newborn and child health (MNCH) services are needed to track and ensure that programmes effectively support the continuum of MNCH care services over time and across wide geographic areas.1 ,2 Health information systems (HIS) are designed to integrate information processes and management structures that inform decisions relating to programme management and healthcare delivery.3 The Performance of Routine Information Systems Management (PRISM) framework identifies key HIS processes as: data collection, transmission, processing, quality checking, analysis, display and feedback.3 ,4 Evaluations of HISs across these processes have highlighted several challenges including poor communication between health system stakeholders, unavailability of individual data at district and national levels and lack of resources to support timely and tailored use of large volumes of data.5 ,6

Many MNCH surveillance programmes have attempted to address inadequacies in health data collection, analysis and use by pairing community health workers with mobile phones to strengthen the ability of the HIS to follow women and children across time and beyond the clinical setting.7–10 Early assessments of these programmes are promising and suggest that the integration of mobile technologies into HIS may improve the quality of collected data, the timeliness of targeted care and overall management of community health workers.11–13 Yet questions remain regarding user perceptions of the ability of mobile technologies to improve HIS data processes in resource-poor settings as compared to traditional paper-based mechanisms.

This study used qualitative interviews and focus group discussions (FGDs) to examine stakeholders’ perceptions of the advantages and disadvantages of a mobile-based health information system (mHIS) versus a paper-based system within the context of MNCH monitoring and evaluation. Specifically, we examined user perceptions of the Mobile Nutrition Day Care Center (mNDCC) system, an mHIS used by the quasi-governmental Society for Elimination of Rural Poverty (SERP) to monitor MNCH services and health outcomes in Andhra Pradesh, India. Prior to implementation of the mHIS, community health workers called Health Activists (HAs) collected data on MNCH services and health outcomes among pregnant and recently-delivered women using a paper-based HIS. Our objective was to understand the perceived effects of the mHIS as compared with the paper-based HIS according to the HAs, who are front-line HIS users, as well as other users in the health system and community. Implications for integrating mobile technologies to improve usability and quality of routine MNCH monitoring are discussed.

Methods

Design and setting

This study was conducted in the state of Andhra Pradesh, which has a population of nearly 50 million. MNCH and nutrition outcomes have been found to be consistently worse for those who live in rural areas, those who are a member of the marginalised groups like Scheduled Castes and Scheduled Tribes, and those with less education and money.14–16

SERP has been providing health services to address deficits in careseeking and population health status in AP's districts and tribal areas since 2003. Throughout the period of this study, mHIS and paper-based HIS were used in parallel by 1732 district HAs and 732 tribal area HAs. Administrative, enrolment and health data on women and children who were eligible for or enrolled in NDCC services were collected through mobile-based and paper-based HISs.

HISs: mobile-based and paper-based

Detailed depictions of the mobile-based and paper-based HIS are shown in figures 1 and 2, respectively. The mHIS depended on HAs for collection and transmission of MNCH data through mobile phones. All users, with the exception of Community Organisation Leaders, received an initial 3-day training on the mHIS. All HAs received two-day refresher mHIS trainings each month. HAs were responsible for enrolling pregnant and recently-delivered women and their children into NDCC services and providing site-based and home-based health education sessions. During their sessions, HAs collected MNCH data on paper registers, then entered and transmitted those data through mobile devices to the SERP servers each day. Transmitted MNCH data were then immediately displayed in online reports consisting of data tables and dashboards. HA Trainers checked these reports for data accuracy and completeness and provided additional informal training to HAs over the phone or during site visits based on the errors identified. The online reports were also printed by HA Trainers and distributed to HAs for sharing with community organisations and NDCC-enrolled women. As part of the data transmission process, the mHIS additionally checked for missing data and sent alerts to District and Tribal Area Officials’ mobile phones. Finally, the mobile system identified NDCC-enrolled women and children in need of upcoming MNCH services and distributed alerts to HAs’ mobile phones to prompt them to organise health education sessions and home-based visits.

Figure 1

Mobile health information system data flow map.

Figure 2

Paper-based health information system data flow map.

Prior to the implementation of the mHIS, HAs collected MNCH data using paper registers only. Within the paper-based system, HAs transmitted data by bringing their registers to HA Trainers each month. HA Trainers would check for errors prior to aggregating and entering the data into computer-based spreadsheets and emailing them to District and Tribal Area Officials. These officials would then aggregate the data from their districts or tribal areas and email those spreadsheets to State Officials. Only aggregate data were available for use and review because women's individual data were stored in paper registers accessible only to the HAs.

Data collection

To examine user perceptions regarding the advantages and disadvantages of both HIS, data were collected in October and November 2014 through interviews and FGDs that solicited users’ successes and challenges during the implementation of the mHIS and the paper-based HIS, including the perceived effectiveness of both systems. Four semistructured interviews (n=4) were conducted with purposively selected State Officials who were responsible for the design and implementation of both systems. We also conducted 13 FGDs with purposively selected HAs (n=30), HA Trainers (n=21), District and Tribal area Officials (n=3) and Community Organisation Leaders (n=28). All interviews and FGDs were conducted in the local language, Telugu, by local facilitators with experience in conducting qualitative research. The number of participants in each FGD ranged from 3 to 8 and lasted approximately 90 min. Information on participants’ area (district or tribal), position, gender and education was also obtained (table 1).

Table 1

Interview and focus group discussion respondent data (n=86)

Analysis

All interviews and FGDs were audio recorded. Two study team members proficient in Telugu and English then translated and transcribed the interviews and FGDs. Open coding was performed to create initial codes that focused on comparisons between the mHIS and paper-based HIS. Following discussions with the study team, a codebook was then applied to all transcripts using RQDA software. The final codes were structured according to the seven key HIS data processes identified in the PRISM framework: data collection, transmission, processing, analysis, display, quality checking and feedback.3 ,4

Results

Within the context of the PRISM HIS processes (ie, data collection, transmission, processing, analysis, display, quality checking and feedback), several important findings emerged regarding the advantages and disadvantages of the mHIS versus paper-based HIS.

Data collection

For collecting routine MNCH data in the district and tribal areas, the most commonly perceived advantage of the mHIS was the shift to documenting data in real time at the point of service delivery. Paper-based HIS users reported that MNCH data were often documented several days after the health service encounter. This led to delays and sometimes to omitted data documentation. In comparison, the mHIS reduced the need to document data during the evening or the next day, as was customary when using the paper-based system.When using paper registers, we may think that we can write down the data a day or two later. But mobile data has to be sent immediately and that's why we now send correct data. (HA, district)The entire [information system] at SERP was designed to capture the data [at the point of delivery]. None of our systems have data that is entered in the evening or the next day, which is the normal way that government systems work. (State Official)

However, the mHIS was not viewed as entirely effective in minimising errors at the point of data collection. Users noted that they were better able to re-enter and correct errors on registers using the paper-based system because it did not require technological knowledge.Unlike with the paper system where we can write it again if we make a mistake, if we have a problem it is difficult in the mobiles. As this is a technical problem, we do not have enough knowledge. (HA Trainer, district)

Data transmission

The mHIS was also viewed as providing greater flexibility and efficiency in how MNCH data were transmitted from one level of the health system to another. HAs commonly mentioned that though the mHIS still required collection on paper registers, the system allowed them to transmit the data without carrying those paper registers outside of the village.Through the mobiles, [the HA] can send data from anywhere, but with papers it has to be sent to the sub-district level …Unless the papers are brought from one place to another, it cannot be sent. (Community Organisation Leader, tribal area)

Yet, in some cases, HAs reverted to using the paper-based system when transmission by mobile phones was not possible due to the phones running out of power or lacking a network signal. While the mHIS allowed data to be entered and saved for transmission once a viable network signal was available, having mobile phones that were fully charged and connected to a network signal was viewed as a limiting factor in the successful transmission of mobile HIS data.If [we] used papers, there won't be any difficulties. We can just write in the paper records from time to time. And paper records don't need to be charged.’ (HA, district)Only if there's full signal does the data get sent. That's why we write the data in the paper register, and then we enter them into the mobile phone and save. When we get a full signal again, we will send the data.’ (HA, tribal area)

Data processing

The mHIS additionally enabled better processing of data, such as reducing the need to manually aggregate health information at each level, which introduced multiple opportunities for error. Compared to paper registers which were easily misplaced or damaged by the rain, the mHIS was described as providing a more durable platform for processing HIS information for both data aggregation and storage.The paper records may get wet in the rain or may tear. But in the mobile-based system, the info will get deleted only if we delete it ourselves. (HA, tribal area)With the paper-based system, somebody writes on a piece of paper which then goes to another person who enters the data, allowing errors to creep in at every level. (State Official)The number of errors introduced depended on the skills of the [HA] Trainer. With [the mHIS], once the data is transmitted, manual totaling and other processing are not required. (State Official)

Data quality checking

Users reported that the mHIS data quality algorithms, which checked to ensure that data were in the allowable range of values and were of the right type and length, were useful because they automatically detected errors and alerted HAs during data collection. Within the paper-based HIS, errors were usually identified several weeks later with no immediate alert. However, HAs noted that the mHIS still required entry of accurate information because some errors were undetectable by the algorithm and could negatively impact the timing of mobile-based service reminders.With the mobile system if we make any mistake, we get an [alert] in return and we can then correct our mistake. But with the paper system, we don't get anything in return. (HA, tribal area)Whatever we enter in the mobile has to be accurate. Suppose we realize that we don't have…a woman's last menstrual period [date] so we [incorrectly] estimate it. By the time the mobile-based system reminds us, we would have missed the data about her delivery. (HA, district)

Data analysis

Data analysis was reported as limited with the paper-based HIS because only aggregate data were available to higher-level officials. Users most commonly praised the capacity of the mHIS in enabling them to analyse data at either individual or aggregate levels. They were able to see general trends and, unlike the paper-based HIS, could also identify specific NDCC beneficiaries in need of services and particular HAs in need of additional training. Individual data also allowed higher level officials to recalculate numbers and thus detect errors that had passed the data quality checking algorithms.We have information…in the form of totals so we are not able to identify the beneficiaries [with the paper-based system]. As the information is by member [with the mHIS], we can know what services individual beneficiaries received each month. (HA Trainer, district)With the paper-based system, what [the district officials] send is one number. We don't know how that figure was calculated… Whereas mobile data we could…break it down further and go to the individual level. (State Official)

Data display

Data display and dissemination was limited to HAs and their supervisors under the paper-based system. In contrast, the mHIS made tables and reports publicly available online. However, users noted that mobile software or computer knowledge was required to take advantage of these mHIS display features.With the paper-based HIS, only we and our supervisors used to know [the information from the reports]. But when we send data through the mobiles, anybody can open and see [reports]. (HA, tribal area)[To make use of the online reports], we need to have some knowledge about computers and the Internet. Trainings are needed. (HA Trainer, district)

Enabling HAs and community members also to see these reports required paper-based printing and reading which was not always feasible. This ultimately limited the utility at the community level of the display feature available through the mHIS.Community members are supposed to use that report…at the village level. But there are some problems like who will print it out or read it to the community members. So there are ways for community members to see the data, but the problem is the usage. (State Official)

Feedback

A final finding related to the timeliness in which data could be fed back to HIS users and community members for decision-making. mHIS users particularly valued receiving MNCH data trends for their village on a daily or weekly basis. These trends were then quickly incorporated into programme planning for home-based outreach provided by the HAs.With the data that we send through paper once a month, we would see our [village's progress] after maybe 6 months. But with the mobile we can know regularly and quickly what our status is. (HA Trainer, district)In the paper system, we used to only spend the last week of the month—right before the review meeting—consolidating, analyzing, and drawing some district-wise inferences to give feedback. Now, with the mobile system, we follow up on the data everyday. (State Official)If we get a report about [immunisation] health services, we can directly go to a woman and… show her the report. Then she will bring her child to get the immunisations. (Community Organisation Leader, district)

Discussion

Our findings showed that front-line and system-level users perceived that the mHIS better supported the MNCH continuum by improving the quality, accessibility and use of HIS data. The mHIS integrated multiple paper registers into an easily transportable system for real-time data collection. It also minimised the administrative burden of transmitting and processing HIS data. Form validation and mobile alerts with service reminders additionally reduced input errors and improved the ability of HAs and their supervisors to identify problems early on. Furthermore, the availability of online dashboards and printable data tables, although irregularly accessed, showed that there was potential for data from the mHIS to be displayed to community members who could not access the data through the paper-based system.

Despite these advantages, our study found several challenges relating to resource and infrastructure limitations that hindered efficient use of the mHIS over time. Lack of electricity and weak mobile network coverage often delayed data transmission, and led to duplication of efforts because HAs sometimes made special trips to areas with better network coverage to transmit HIS data. Similarly, limited access to computers and printers at the village and district levels limited Community Organisation Leaders’ ability to access and use the mHIS data. These resource and infrastructural challenges collectively may also have exacerbated the ‘digital divide’, especially in tribal areas.17 ,18

However, as compared to a paper-based HIS system, this study suggests that mobile phone technologies can improve health information processes in resource-poor settings. Our results align with previous research studies that have shown that HAs are better able to reach women for time-dependent MNCH services using a mHIS rather than paper-based HIS, which ultimately improves the continuity of care.19 Improved MNCH care was attributed to timely analysis and feedback to HAs through mobile alerts, more regular monitoring of indicators by higher level officials, and the ability of all mHIS users to access individual records to investigate problems encountered by specific individuals or service centres.

More efforts are needed to ensure sufficient training and use of mHIS data at all levels of the HIS in low and middle-income countries.4 ,20 ,21 Although HAs reported positive perceptions of the mHIS, our study highlighted their limited technological understanding of the mHIS as compared to the paper-based HIS. Other studies have shown that despite high reported acceptability, there may be low use of mobile phone features such as the lack of user follow-up on mobile alerts as a result of technological barriers or omissions.9 ,22 In addition, since mHIS system-generated data (ie, such as time stamps or number of times supervisors checked reports) were not tracked, actual usage could not be compared to reported usage.22 This omission made it difficult to assess actual uptake and use of specific components of the mHIS. Our findings suggest that integrating user metrics with trainings tailored to users’ technological background and resource availability may improve the usability of mHIS at all levels of the healthcare system.

Finally, the study's limitations are worth noting. Our research did not explicitly address users’ perceptions of health data privacy, which may have influenced perceptions regarding the mHIS. Some findings may also have been limited by recall and social desirability biases towards a new technology. Despite these limitations, this study demonstrated that mobile phone technologies can potentially enhance data quality and user engagement in MNCH health information processes, provided there is adequate training, customisation and supervision. Increasing financial investments and stakeholder engagement at village, district and state levels may further enhance the potential of mobile-based HISs.

Acknowledgments

The authors would like to thank all the study participants who made this research possible, including a special thanks to the staff of the Society for Elimination of Rural Poverty (SERP) and Community Consultants for their critical input during instrument design and data collection: Dasari Sandhya, Kuruvanapalle Sudha Rathna, Sarakana Hymavathi, Tadi Lakshmi, Patti Rejeswari and Nandipalli Venkata Satya The authors also extend their sincere thanks to Vijeta Rao and Jack Palmieri, who participated in data collection, translation, transcription and analysis. The authors are likewise grateful to Bhargavi Anreddy and Aleem Mohammed for their assistance during data collection and to Manoj Aravind for his help during data collection, translation and analysis.

References

View Abstract

Footnotes

  • Contributors LHN and LDC designed the study. LHN also conducted the analysis and wrote the first draft of the manuscript. AEL, LJ and SA advised the study design, participated in data interpretation and revised the manuscript. All authors have read and approved the final manuscript.

  • Funding This study was funded by the Society of Elimination of Rural Poverty, the Johns Hopkins Global mHealth Initiative, and the World Health Organization Department of Reproductive Health and Research with funding from the United Nations Innovations Working Group Catalytic Grant Mechanism for mHealth Scale managed by the United Nations Foundation.

  • Competing interests None declared.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data sharing statement Additional unpublished portions of the translated focus group discussion and interview transcripts are available through the Society for Elimination of Rural Poverty. Interested parties can contact Lakshmi Durga Chava, the Director of the Human Development Unit, for access to these transcripts.

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.