Article Text

Download PDFPDF

Rapid phase I evaluation of a novel automated hand hygiene monitoring system in response to COVID-19
  1. Katie-Rose Cawthorne1,
  2. Darren Powell2,
  3. Richard PD Cooke1,2
  1. 1Innovation Department, Alder Hey Children's NHS Foundation Trust, Liverpool, UK
  2. 2Department of Laboratory Medicine, Alder Hey Children's NHS Foundation Trust, Liverpool, UK
  1. Correspondence to Ms Katie-Rose Cawthorne, Innovation Department, Alder Hey Children's NHS Foundation Trust, Liverpool, UK; katierosecawthorne{at}

Statistics from

Summary box

What are the new findings?

  • A novel automated hand hygiene monitoring system, Hy-genie Lite (HL), was successfully implemented in a busy pathology department during the COVID-19 pandemic.

  • The system proved highly accurate with a sensitivity and positive predictive value of 99%.

  • Using HL, hand hygiene (HH) performance improved by 14.6% during a 6-week intervention period following group staff feedback. There was no significant change in pathology workload during the evaluation period.

How might it impact on healthcare in the future?

  • The full Hy-genie system is designed to give individual staff personalised feedback on their own HH performance with realistic improvement goals. This early study suggests that further evaluation of the complete system is warranted in a high-risk clinical environment.


Direct observation (DO) of hand hygiene (HH) behaviour remains the gold standard tool for measuring staff compliance during the COVID-19 pandemic. However, gathering HH data in the current environment may be challenging for many healthcare facilities due to resources being diverted to COVID-19 containment measures. Hence, audit on HH compliance may be severely compromised due to lack of labour force to perform DO. This is problematic as hospital transmission of COVID-19 is high.1

Automated hand hygiene monitoring systems (AHHMS) have been developed in recent years to enable healthcare organisations to gather robust HH data with minimal investment of labour. Group monitoring systems and badge-based systems are the two most common types of AHHMS available in the marketplace.2 Group monitoring systems track usage of HH dispensers to give an idea of HH frequency by staff groups. Badge-based systems typically require healthcare workers (HCWs) to wear an additional tracking device that communicates with dispenser-based sensors.

Hospitals with AHHMS already in place prior to the COVID-19 pandemic are in an advantageous position. An AHHMS was used to capture 35 million HH opportunities between January and May 2020 at the height of the pandemic.3 Capturing a similar number of HH opportunities via DO would not be feasible. Makhni et al4 used the same AHHMS to demonstrate that in the early days of the pandemic, HH compliance reached 100%, although unfortunately it did decline to 51.8% within a few months.

AHHMS which purport to measure HH compliance rates are typically US-based and are reliant on healthcare institutions having a high proportion of single-patient rooms. These AHHMS focus on single room entries and exits as surrogates for WHO HH moments 1 and 4, that is, washing hands before and after patient contact.5 Such an approach contributes to a limited picture of true HH behaviour.6

After an extensive consultation exercise with NHS HCWs,7 the concept of a novel AHHMS, ‘Hy-genie’, has evolved.

The end result of any HH initiative is to prevent healthcare-associated infections (HCAIs) and improve patient safety. Simply encouraging HCWs to perform HH more often could be a simpler improvement initiative for HCWs to work towards. The Hy-genie approach provides HCWs with HH activity feedback with improvement goals (based on a 20% overall improvement goal from baseline). Such an approach is simple and easily understood by staff and warrants further evaluation.

Here, we describe the first-phase evaluation of the Hy-genie system. The aim was to validate the accuracy of two key components of the system (Hy-genie Lite (HL)) in a low-risk clinical environment using established epidemiological statistics.8 Following this, the feasibility of a group HH feedback intervention was assessed in a busy NHS pathology department.


System: HL

Hy-genie has been developed by an NHS consultant medical microbiologist (RPDC, Hand Hygiene Solutions) with support from an innovation programme based in an acute NHS hospital in England. In response to the COVID-19 pandemic, an early phase of Hy-genie was launched, namely, HL. HL is a group monitoring AHHMS which contains two components: a pickup sensor and a base station (figure 1). HL captures the usage frequency of soap and gel dispensers. The full Hy-genie system will be a badge-based AHHMS enabling dispenser usage to be linked to individuals, but this was not evaluated in this study.

Figure 1

Components of Hy-genie Lite. Pickup sensors are small battery-powered devices that are mounted beneath soap or gel dispensers and use capacitance sensing to detect operation of the dispenser. When usage of the dispenser is detected, a time stamped message is sent to a central server via the nearest base station. The base stations act as routers for the messages sent by pickup sensors. Data stored in the base station are then visible on a computer dashboard. The daily counts of the number of uses of each dispenser, as represented by the counts of the number of sensor messages, were analysed during this study.

HL aims to provide novel and simplified HH feedback. The system captures a daily frequency of HH events performed by a staff group. A performance target is then set by management and then weekly reports demonstrate whether the staff group has achieved its target. A 20% HH performance goal for group feedback was based on previous modelling of improved HH compliance linked to HCAI reduction.9 A goal setting approach to promote HH in hospitals has been previously recommended,10 and during consultation with users, it was deemed that performance goals were easier to action than users having to interpret raw HH activity data themselves. An example of a weekly performance report is presented in figure 2.

Figure 2

An example of the weekly report shared with study participants via email and in daily morning safety huddle meetings as part of the study intervention. The traffic light system is based on target levels of HH activity compared with data collected in the baseline period. Red indicates ‘performing below expectation’; HH activity is lower than baseline period for that day. Amber indicates ‘performance improving’; HH activity is between 0% and 20% higher than baseline period for that day. Green indicates ‘improving above expectation’; HH activity is more than 20% higher than baseline period for that day. HH, hand hygiene.

Evaluation setting

An evaluation protocol was registered with the Trust’s audit department as a quality improvement initiative. The pathology department was chosen as a COVID-19 low-risk unit. Study participants were the 102 pathology staff working across the five different specialty disciplines in a single-site, open-plan laboratory. The Trust’s HH policy for this department recommends that staff should perform HH on entry and exit of the laboratory areas, when hands are visibly soiled and after removing gloves.11

An initial review of the department was performed to ensure that adequate soap, gel and moisturiser dispensers were available for staff use. This was performed in conjunction with the Trust’s infection prevention and control (IPC) team. The Trust’s Innovation Executive Lead formally endorsed the evaluation, ensuring that any clinical governance issues would be dealt with at a senior management and operational level. Prior to implementation, the development team performed in-house quality assurance testing to ensure correct functioning and accuracy.

Installation of HL

A written proforma detailing HL installation instructions was presented to the estates department. Nineteen HL pickup sensors were provided for installation covering 11 soap and 8 gel dispensers within the pathology department. The 19 pickup sensors were wirelessly linked to six base stations, which provided connectivity to the IT network over a secure wifi connection. Throughout the evaluation, a daily review of all dispensers was undertaken to assess if any dispensers were empty or faulty.

Accuracy study

The accuracy of HL in the pathology department was validated by determining its ability to correctly identify an HH event. HH events detected by the gold standard method (DO of HH by a trained auditor) were compared against data produced by HL over a 6-week period. Using a standardised proforma, the auditor (RPDC) undertook a number of DO HH events himself, each of which were timed and dated. These events were then compared with HL’s HH activity database. Accuracy of HL was assessed by calculating sensitivity and positive predictive value (PPV).

Interventional study

Following the accuracy study, HL was run in the background for 6 weeks to establish baseline HH activity in the pathology department. A behavioural interventional study was undertaken following the baseline period. The aim was to determine if the intervention (ie, weekly performance reports sharing group HH data alongside realistic targets) leads to an increased frequency of HH events.

During the 6-week intervention period, weekly performance reports (figure 2) were cascaded to all pathology staff by email using existing governance structures. A brief interpretative comment was also included with each performance report. Hy-genie performance reports were also included in each pathology specialty’s daily morning safety huddle.

A quality assurance scheme was introduced to ensure continuing accuracy of the HL system. Three times per week during the 12-week study period, a designated member of staff agreed to record the date and time of five HH events (DO), and this was compared with the HL database.

Staff perceptions

Occupationally acquired dermatitis

A questionnaire was sent out to all participants to assess likely rates of occupationally acquired dermatitis among staff prior to the introduction of HL. This was based on the WHO’s hand and skin self-assessment tool.12 A hand condition score of <23 was considered poor. This survey was repeated at the end of the evaluation.

Staff attitudes to HH monitoring and the role of innovative technologies

At the beginning of the evaluation, a questionnaire was sent to all participants to assess their views on DO and innovative technological approaches to HH compliance monitoring. Questions used were identical to those used in a previous study.7 This survey was repeated at the end of the evaluation.

Statistical methods

Libreoffice Calc was used to perform χ2 tests for HH activity preintervention and postintervention, daily and weekly HH activity, and pathology workload data. The number of monthly specimens processed were controlled to ensure that changes in HH activity could not be attributed to changing workload.

Patient and public involvement statement

Patients were not involved in the design, conduct, reporting or dissemination plans of this study.


Overall, 19 889 HH events were captured throughout the evaluation, including the accuracy study, baseline and intervention periods.

Installation of HL

Operational issues arose that were not accounted for in the installation proforma. The HL base station is internet enabled, yet the network coverage within the pathology department was poor. The network coverage was improved after involvement from senior management.

In the installation proforma, it was advised that the pickup sensors were secured to the soap and gel dispensers by screws. During installation, tape was used instead to secure the pickup sensors which meant they were at risk of falling.

Accuracy study

A total of 219 DOs were assessed against HL across a 6-week period. HL identified 217 HH events. HL failed to detect two HH events (false negatives) and detected two false positive HH events. Sensitivity (219/221) and PPVs (219/221) were both 99%.

Interventional study

Excluded data

Data from weekends and bank holidays were excluded from analysis and weekly performance reports. Initially during the intervention period, data for all 7 days were shared with study participants. However, after only 2 weeks of providing feedback to staff, it was apparent that there were very small numbers of HH events on weekend days compared with weekdays (mean 47 and 232, respectively). Due to the concern that this information could be misleading to staff, weekday and weekend data were separated from week 2 of the intervention period. Only weekday performance reports were presented to staff as part of the intervention.

One soap dispenser became faulty, constantly dripping onto the associated HL pickup sensor. As a result, this generated a large number of anomalous false positive results which were quickly identified from HL’s database during the intervention period. A review of the previous validation and baseline data indicated that this problem only occurred during the intervention phase of the evaluation. This soap dispenser unfortunately was not able to be repaired until after the evaluation period was over. Therefore, all data captured by the associated sensor were excluded from the analysis. This equated to approximately 60 HH event recordings per day.

Data reported in this results section excludes data from weekends and bank holidays and data from the faulty soap dispenser.

Baseline data collection period

A total of 6956 HH events were recorded across 30 days of data collection on weekdays. The mean weekday average HH events per weekday was 232. There was no significant difference between the daily HH events from Monday to Friday: χ2 (5, n=6956)=4.81, p=0.44. Similarly, there was no significant difference from week to week: χ2 (4, n=6956)=3.48, p=0.48.

Intervention period

Total HH events recorded over the 6-week intervention period was 7441. The average HH events per day during the intervention period was 266. This represents a 14.6% improvement (34 more HH events per day) during the intervention period compared with the baseline period.

This rate of improvement was statistically highly significant: χ2 (27, n=13 902)=147.80, p<0.01.

The quality assurance scheme proved difficult to achieve due to staff workload pressures. Quality assurance was undertaken only on five occasions (14% of occasions). There was complete concordance between DO and HL results on all occasions.

Pathology monthly workload

There was no significant difference in workload during the 12-week study period (between April and June 2021): χ2 (9, n=44 443)=2.32, p=0.98.

Staff perceptions

Occupationally acquired dermatitis

The pre-study questionnaire received 35 responses (34%). Nine respondents had a poor hand condition score. Six staff attributed their poor score to the soap or gel HH products available. The post-study questionnaire received only eight responses (8%) and was not analysed further.

Staff attitudes to HH monitoring and the role of innovative technologies

The pre-study questionnaire received 19 responses (19%). All respondents agreed that HH compliance was important (Likert score of 4 or 5). All respondents rated their own HH compliance highly (Likert score of 4 or 5), whereas only 79% of respondents rated colleagues’ HH compliance highly (Likert score of 4 or 5). Free text responses to staff perceptions of DO demonstrated that respondents did not have an opinion on this method of HH monitoring. When respondents were asked about their views on an HH monitoring system that could provide personal, objective and timely feedback on HH performance, 74% of respondents felt this would be useful (Likert score of 4 or 5), and 47% would feel comfortable with this (Likert score of 4 or 5).

Only five responses were received post-evaluation and were not analysed.


HH in pathology

HH monitoring is not routinely conducted in a pathology department; therefore, there is no pre-study HH compliance data with which to compare our results. This is reflected in the pre-study questionnaire indicating that pathology staff do not have strong views on DO, unlike those HCWs surveyed in a previous study.7 Compared with this previous study, pathology staff felt less comfortable with the idea of an innovative HH monitoring technology than other HCWs (47% of staff compared with a mean of 57.5%). Additionally, two-thirds of pathology staff with poor hand condition attributed this to soap products provided by the hospital. HH improvement strategies could be difficult to implement in this staff group without first addressing concerns about occupationally acquired dermatitis.


There has not been a formal review of the accuracy of different AHHMS, but comparison against individual studies shows that this accuracy study is the largest completed. In previous AHHMS validation exercises, 153, 58 and 123 DOs were used as the validation gold standard,13–15 compared with 219 DOs in this study. HL is also among the most accurate systems available.

The WHO Hand Hygiene Implementation Guide and an analysis by Yin et al suggest that hospitals should observe between 150 and 200 HH opportunities per nursing unit per time period to obtain reasonable estimates of adherence rates when performing DO HH audits.16 We believe that this figure of 150–200 DO could also be applied to AHHMS validation.

An accurate AHHMS will provide robust quantitative data on HH performance. However, this must always be incorporated into a multimodal HH promotion programme to realise their full potential.17

Implementing an innovative system

It was challenging to keep NHS staff motivated during this study, demonstrated by the very low response rate to the staff questionnaires. By its very nature, evaluating innovative technology is outside of the boundaries of normal practice. A recent report shows that NHS workers are burnt out as a result of the pandemic.18 Participating in this evaluation may have placed an additional demand on an already stressed workforce. This further supports the need for innovative technology like Hy-genie to streamline NHS workload and reduce the burden of manual data collection.

Both the innovation team and the pathology department were involved in the design of the study, but the Trust’s estates department responsible for installing the system was provided only with a short briefing. As a result, there were significant delays throughout the installation process. This reflects the importance of wider stakeholder involvement in future such evaluations. A nominated stakeholder from all parties should be defined at the start of implementation to ensure that timelines are kept to.

Finally, dispenser faults in this evaluation meant that a significant volume of data was lost, as evidenced by the faulty dispenser. This issue highlighted the need to have a quality assurance mechanism to identify equipment faults and respond quickly.

Evaluation of intervention

The weekly performance reports were successfully disseminated to study participants, indicating that there are no technical barriers to delivering such an intervention in a busy NHS hospital. During the intervention phase of the study, there was a 14.6% increase in HH activity (p<0.01). This provides promising evidence that simplified target-driven feedback on HH frequency could improve HH performance, especially as our evaluation did not include a multimodel IPC improvement strategy.

The weekly performance reports initially included HH data from all 7 days but were later changed to just include weekday data. Following this change, the trend to improving HH performance did not alter significantly. However, in future implementations, baseline data should first be reviewed to identify extraneous results before beginning the interventional phase.

The encouraging results in this evaluation are supported by other studies. Two studies by Armellino et al19 20 demonstrated the effectiveness of staff feedback on HH performance using regular email performance reports; one showed a rise from 6.5% to over 85% compliance for a medical ICU.19 Regularly providing such performance reports using DO data alone would not usually be feasible in a pathology department as they do not have an established HH audit programme. However, the use of AHHMS to facilitate such an intervention could be useful.

Future developments

An unobtrusive AHHMS is essential to ensure staff workflow is unimpeded. A study by Hess et al showed that use of a badge-based AHHMS was reduced during COVID-19.21 AHHMS are designed to reduce workload, not to increase it;21 therefore, providing personalised data must be balanced with additional demands placed on staff to wear a badge-based system. The full Hy-genie system aims to give personalised HH performance insights via an adapted badge holder, the ‘beacon’. As the beacon will carry the existing staff ID badge, this should reduce the burden on staff to further change their behaviour. The next development phase of Hy-genie is to test all three components of the system in a high-risk clinical environment.


The high accuracy of a novel AHHMS (HL), combined with an improvement in staff group HH performance, suggests that further investigation of the complete Hy-genie system (involving personalised staff feedback) in a high-risk clinical environment is warranted.

Ethics statements

Patient consent for publication

Ethics approval

This study involves human participants but was not approved by an ethics committee or institutional board (no reference number or ID). The Hy-genie evaluation protocol was widely discussed with colleagues within the Trusts’ departments of Innovation, Research and Development and Laboratory Medicine. There was general consensus that the evaluation was a quality improvement initiative and that ethics committee approval did not need to be sought, especially as all hand hygiene data collected were anonymised. The evaluation was, however, registered with the Trust’s audit department. The evaluation involved anonymised staff group feedback and not individual feedback. As no individual members of staff could be identified, individual staff consent was not considered necessary or appropriate.


The authors acknowledge the support and encouragement of the pathology staff at Alder Hey Children’s NHS Foundation Trust during the evaluation period.



  • Contributors All authors contributed to the design, analysis and interpretation of the findings. The first author wrote the initial drafts. All authors approved the final submitted version.

  • Funding This project was funded by the Innovation Department at Alder Hey Children’s NHS Foundation Trust and Hand Hygiene Solutions Limited.

  • Competing interests K-RC and RPDC report personal fees from Hand Hygiene Solutions Limited. RPDC is also a director of Hand Hygiene Solutions Limited.

  • Provenance and peer review Not commissioned; externally peer reviewed.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.