Article Text
PDF
Abstract
Objective To describe and compare safety and usability between a peristaltic large-volume intravenous smart pump (IVSP) and a novel pneumatic large-volume IVSP during clinical use.
Methods A prospective, comparative study was conducted in a large, tertiary hospital in the southeastern USA. Safety and usability were measured by observation during medication administration (medication administration error, interruptions, programming time), dose error reduction system (DERS) compliance, end-user surveys and compliance with manufacturer setup requirements. Study implementation began on a small pilot unit for 1 month, followed by data collection on the study unit over 2 months.
Results For the observed medication administrations (N=158): 79 peristaltic (36 primary; 43 secondary) and 79 pneumatic (42 primary; 37 secondary), use of the peristaltic IVSP was associated with significantly (p<0.05) higher medication administration errors and programming time (11.9 s) and a significantly higher number of interruptions during programming.
DERS compliance was significantly less (p<0.001) with the peristaltic (75.9%) as compared with the pneumatic IVSP (99.8%). Programming workload (National Aeronautics and Space Administration Task Load Index) was significantly (p=0.004) higher with peristaltic versus pneumatic IVSP, and the usability (System Usability Scale) was significantly (p=0.007) lower with peristaltic versus pneumatic IVSP. There was a 0% compliance with peristaltic secondary setup requirements in 43 observed infusions.
Conclusions Though nurses had a high level of experience with the peristaltic IVSP, results of this study support that the pneumatic IVSP was easier to use and associated with fewer errors and deviations from safe practices as compared with the peristaltic IVSP.
- delivery of health care
- nursing
- patient care
- pharmacology
Data availability statement
All data relevant to the study are included in the article.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Summary box
What are the new findings?
This is the first comparative IVSP usability study conducted in an actual clinical setting and represents real-world clinical evidence.
Numerous clinically relevant differences were found in both the usability and safety between these two types of IVSP technologies.
In all measures, both safety and usability was improved with the pneumatic IVSP as compared to the peristaltic IVSP.
How might it impact on healthcare in the future?
Results from this study on safety and usability between these two types of IVSPs during actual clinical use is important to better inform end-users and decision-makers who rely on these fundamentally important devices to provide life-saving patient care.
Innovations in IVSP is important for improving the usability and safety of intravenous medication administration.
Introduction
Intravenous infusion pump systems are among the most frequently used technologies in healthcare with an estimated 90% of hospitalised patients receiving intravenous medications via infusion pumps, particularly in critical and acute care settings.1 To overcome limitations posed by manual calculation of intravenous medications and fluids, intravenous smart pumps (IVSPs) include drug libraries and dose error reduction systems (DERS) to provide users with automatic dose calculations and guidance. Clinical use of IVSPs with DERS began at Massachusetts General Hospital in 1996 and has become widely accepted as a standard of care for the reduction of infusion-related medication error.2 While the use of IVSPs has been associated with reductions in medication error rates, they have not eliminated error.3–6 Furthermore, data do not support that the use of IVSPs has had a measurable impact on decreasing adverse drug events.3 7–9
Common sources of user error include programming errors and overriding the IVSP safety features.6 10–12 The complexity of the device user interface, time constraints for programming and limitations in drug libraries have been described as reasons to bypass IVSP safety features.13 Clinicians in critical care and medical-surgical clinical environments are frequently interrupted and rushed during IVSP programming.14–17 Despite an increasing focus in healthcare on patient safety and quality of care, and improvements in technology, medication errors and usability issues with IVSPs continue to be a significant patient safety issue.18–20
Large-volume IVSP technology
The setup of any IVSP involves attaching the intravenous bag with its tubing to the pump, following directions to programme the amount and rate for fluids to achieve the desired dose, and following the operational instructions provided by each manufacturer. The two most common large-volume IVSP technologies in current use are linear peristaltic head-height differential and pneumatic-driven systems. Peristaltic pumps use gravity by hanging the intravenous bags above the level of the heart to create the hydrostatic pressure needed to achieve accurate flow for both primary and secondary infusions. Data support a lack of clinician knowledge regarding specific setup requirements for this function.21–23 Lack of compliance with manual setup requirements leads to unpredictable, and largely undetectable, variation in flow rates of fluid and medication delivery, especially with secondary infusions.20–22 As a result, the 2020 Institute for Safe Medication Practices guidelines for optimising safe implementation and use of smart infusion pumps21 recommend the use of systems for secondary infusion that does not require head-height differentials.24
In contrast, pneumatic-driven systems use internal flow control valves to independently regulate both primary and secondary flow rates, which function to move fluids forward for infusion into the patient. Because the cassette is not dependent on gravity flow, this approach eliminates the need to manually create sufficient hydrostatic pressure to assure intended fluid flow.22
FDA infusion pump recalls
Over the past decade, the safety and usability challenges associated with IVSPs have resulted in numerous FDA recalls. When recalls occur, they are classified by the FDA into one of three possible classes according to the degree of associated health hazard. Class 1 recalls are the most serious and are defined by the FDA as ‘a situation in which there is a reasonable probability that the use of, or exposure to, a violative product will cause serious adverse health consequences or death’.
A review of the 2020 FDA recall database revealed 10 infusion device-related class 1 recalls, mostly seen with one of the most commonly used peristaltic large volume IVSPs in the US acute care market.22 Given the extent and nature of issues related to administration and safety of intravenous medications via IVSPs, the study was developed to compare a legacy peristaltic system (peristaltic) with a recently FDA-approved pneumatic-driven IVSP.
The purpose of this study was to describe and compare safety and usability measures between a peristaltic large-volume IVSP and a new pneumatic large-volume IVSP during actual clinical use.
Conceptual framework
Using findings from previous research, a conceptual framework (figure 1) was developed to provide the foundation for this study. As outlined in the framework, the study aims were based on two major concepts: safety and usability.
Conceptual framework for intravenous smart pump (IVSP) safety and usability (used with permission from Karen K. Giuliano, PhD, RN). *Dose error reduction system (DERS). **National Aeronautics and Space Administration Task Load Index (NASA-TLX).
Methods
A prospective, pre–post comparative observational and survey study design was used to address the safety and usability research aims highlighted in figure 1.
Subjects and setting
The study was conducted in a large, tertiary care hospital in the southeastern USA over two phases. While the pneumatic IVSP had received 510K clearance by the FDA in June 2019, prior to this study, it had never been used during actual clinical practice. Thus, we selected clinical units with predictable patient types and the fewest competing priorities. The pilot phase of this study was conducted for a period of 1 month in the hospital’s outpatient infusion centre. Once the staff and research team became familiar with the new pump, study data collection was completed over 2 months in an inpatient bariatric surgery unit, where patients remain for the duration of their hospitalisation, and implementation required only a small, consistent number of nursing staff.
Prior to the study initiation, the protocol was approved by the organisation’s Institutional Review Board with expedited review. Nurses in the study reviewed the informed consent form and participation in the surveys, and observations were voluntary.
Patient and public involvement
This study did not include patient or public involvement as it was a study about safety and usability of two different types of IVSPs with nurse subjects only.
Study instruments, aims and measures of IVSPs
At the study hospital, the large-volume IVSP in current clinical use was a peristaltic IVSP (BD/Alaris), and the comparison IVSP was a recently approved pneumatic-driven IVSP (Ivenix). The use of the pneumatic IVSP was limited to those nurses who completed training for use in the study and access was limited using a PIN code. During the period when the pneumatic-driven pump was being used, the peristaltic pump remained available on the study units for use, should any issues arise while using the pneumatic pump or for use by nurses who had not been verified to use the pneumatic-driven IVSP.
Both IVSPs use reporting systems with digital logs for data, although the peristaltic data were not available at the unit level since these pumps were sourced for use from the central supply department. To note, with the exception of one new graduate nurse, nurses in the study had significant experience using the peristaltic IVSP.
The section below describes the aims of the study and the measures for variables included. Table 1 summarises these measures used.
Description of study variables and their measurement for reporting (created by authors)
Aim 1: safety measures
1a. Medication errors: direct observation of nurses while programming and administering intravenous medications and fluids was done to detect errors at the point of care. Reported errors into the event management system for the facility were also reviewed during the study period.
1b. Interruptions: Interruptions during programming of the IVSP was measured by direct observations.
1c. DERS compliance: the IVSP digital logs were used to evaluate DERS compliance during the study periods. The pharmacist dedicated to IVSP use and the biomedical engineering team provided digital data on the IVSP Digital logs and DERS logs for the peristaltic pump on the BD Knowledge portal. However, review of peristaltic IVSP from the study units was not possible, given that the pumps were not stationary on the unit since pumps were removed after use and cleaned in another department. A surrogate assessment for these measures was extrapolated from reports from units of the same level of acuity as the study unit during the same period. An additional direct observation of a subset of nurses was done to assess for DERS compliance during actual clinical use. For the pneumatic IVSP, digital logs were used for these measures specific to the units where the pump was used.
Aim 2: usability
2a. Compliance with manufacturer required setup: direct observation of nurses while setting up secondary infusions was done to evaluate compliance with manufacturer guidelines. For the peristaltic pump, correct application of all elements included head-height differentials in relation to the pump position and unclamping the line for secondary infusions. For the pneumatic pump, no head-height differential is required, and an alarm will sound if the secondary clamp is closed when initiating a secondary infusion.
2b. Programming workload and programming time: The National Aeronautics and Space Administration Task Load Index (NASA-TLX) is a widely used multidimensional assessment tool for the measurement of subjective cognitive workload. Although originally developed at NASA’s Ames Research Center for use in aviation, it has become an important tool in human factor research.25 Part 1 (raw scores) of the NASA-TLX consists of six individual subscales measuring mental demand, physical demand, temporal demand, performance, effort and frustration. Subscale raw scores can be used individually or in combination to assess participant’s cognitive experiences during task performance.25 Each subscale uses an interval scale ranging from low (0) to high (20) and subscales can be summed to create combined scores, with higher scores indicating higher perceived cognitive workload. The NASA-TLX has been used widely in healthcare and for usability testing in the simulated environment and is reliable and valid for cognitive workload, with a reported test–retest reliability of 0.77 and a high concurrent validity (0.73–0.79).26 27
2c. Programming time was measured by direct observation using a stopwatch function on a smartphone.
2d. Ease-of-use: the System Usability Scale (SUS), originally created by John Brooke in 1986, is a widely used and reliable tool for measuring product usability.28 The SUS consists of a 10-item questionnaire with five response options for respondents from Strongly agree to Strongly disagree. Total scores can range from 0 to 100 and higher scores indicate more ease of use and are also translated to an A–F grade. Additional questions about the satisfaction of the use of both IVSPs were asked in addition to the SUS to capture-relevant information about subjects’ experience.
Study procedures
The drug libraries for the pneumatic IVSP were built by the pharmacist at the study hospital who oversees IVSP operations. The drug libraries for the pilot and study units were created to include medications commonly used on each of the study units based on historical data from the peristaltic IVSP. All drug entries in the libraries included hard and soft limits that were reconciled by second pharmacist, the principal investigator and the clinical research coordinator.
Prior to educating the nurses on the pneumatic IVSP, nurses on both the infusion and inpatient units completed the SUS and NASA-TLX surveys to capture their perceptions on the peristaltic pump, since these nurses already had extensive clinical experience with that IVSP. For the pneumatic IVSP, the NASA-TLX and SUS were given to nurses at the end of the postimplementation study period.
For the pneumatic IVSP education, nurse participants reviewed an online education module, followed by hands-on demonstration and practice prior to the clinical use and study data collection. This initial education was provided by pneumatic ISVP vendor. Daily rounds for continued education/review of pneumatic use were completed by the research coordinator, who also completed all observational data collection throughout all phases of the study.
Data analyses
This study was primarily exploratory and descriptive in nature, and included a safety aim with three associated subaims, a usability aim with four associated subaims and various units of analysis. First, descriptive statistics using appropriate measures of central tendency and dispersion were generated for all aims. Additional analyses for each aim were then completed based on the sample size and unit of analyses specific to each subaim, which are described below.
Results
Aim 1: safety measures
Medication Errors
Using convenience sampling, a total of 158 randomly chosen actual infusions were directly observed from the nurse’s entrance into the patients’ rooms until the infusion was started. Of those, 79 were with the peristaltic IVSP (36 primary and 43 secondary) and 79 with the pneumatic IVSP (42 primary and 37 secondary). To determine completion of secondary infusions, the researcher returned at the expected end of infusion of secondary medications to assess delivery status.
Eleven errors were seen during the total number of medication administrations observed (6.9%), all of which occurred with the peristaltic IVSP, which was significantly higher than no errors seen while using the pneumatic IVSP (χ2 5.61 (95% CI 0.9 to 14.71; p=0.0179). Seven (64%) errors occurred during secondary medication administration and four (36%) during primary infusions. As shown in table 2, 8 out of the 11 errors were related to incomplete antibiotic dosing. We used The National Coordinating Council for Medication Error Reporting and Prevention system to rate all errors and rated antibiotic dosing errors as category D.
Medication errors observed in the peristaltic pump (created by the authors)
Interruptions
During the 158 observations, 14 (9%) interruptions occurred during programming on both pumps (n=10 peristaltic; n=4 pneumatic). The mean duration of interruption was 55 s, with a range of 26–133 s. Types of interruptions included phone calls (n=7; 50%), patient requests (n=4; 28%) and staff/physician (n=3; 21%). Of those, 6/14 (42.9%) resulted in a delay in pump programming with the peristaltic pump (range 5.6–45.2 s) and 1 (7.1%) had a dosing medication error with albumin concentration on the peristaltic IVSP.
DERS compliance
The digital databases were queried to evaluate compliance with the DERS. Compliance with the pneumatic DERS was 99.8% on 1779 infusions. The four deviations occurred with a drug that was in shortage and was not in the DERS. Using data from the BD Knowledge Portal from units at the same medical surgical level, acuity during the same period was taken from 54 151 infusions with the peristaltic, compliance was 75.9% (n=41 150). This was statistically significantly different using χ2 comparison of proportion analysis (p<0.0001). Since this was not a direct comparison, additional point prevalence observations for DERS compliance with peristaltic pump were done using a convenience sample of 43 infusions on the study unit and compliance was 41/43 (95.3%), which was significantly lower versus 99.8% found in the pneumatic IVSP (p<0.0001).
Aim 2: usability measures
Compliance with manufacturer required setup
Compliance with peristaltic secondary medication setup requirements was observed in 43 infusions during actual clinical use. To be deemed ‘compliant’ overall, the position of the IVSP, primary and secondary infusions all had to meet manufacturer requirements. In observation of the 43 infusions, none of the nurses was completely compliant with all steps of the process with the peristaltic infusions. Thus, a 0% compliance with the peristaltic system step-up was found, with details summarised in table 3.
Observed measures of compliance (created by the authors)
In addition to these data, 4/43 (9.3%) peristaltic infusions were found to have clamped secondary lines during the infusion, resulting in a dose that was not administered on time. Because there are no head-height requirements for the pneumatic IVSP, and an alarm alerts the clinician when the secondary clamp is closed, set-up was found to be 100% compliant.
Programming workload and timing
Nurses completed NASA-TLX for both IVSPs. A total raw score combining all six elements was used to evaluate differences in the NASA-TLX scores for each IVSP. Total mean scores for the peristaltic was 38.09±24.06 (n=23) and 20.06±13.6 (n=18) for the pneumatic. The Mann-Whitney U test found that the programming workload on the peristaltic was significantly higher versus pneumatic (p=0.013), with additional details provided in figure 2.
Mean scores from individual elements of National Aeronautics and Space Administration Task Load Index (NASA-TLX).
During the observed infusions (N=158), programming times in seconds were measured in those without interruptions (n=136) for both IVSPs for primary and secondary infusions. The times measured were significantly shorter in duration with the pneumatic IVSP on total, primary and secondary infusions (table 4).
Comparison of programming times for primary and secondary infusions (created by the authors)
Ease-of-use
Subjects completed the SUS measures for both IVSPs. Three cases had one missing data point (peristaltic n=1; pneumatic n=2); thus, we imputed mean scores for those data points for final calculation of mean SUS. An independent sample t test showed that the pneumatic usability scores (n=18; mean 77.16±13.18) were significantly higher versus the peristaltic (n=23; mean 61.68±17.95) (p=0.004).
Discussion
In this study, we found significant differences in the safety and usability measures between the peristaltic and pneumatic IVSPs. Fewer use-errors were found during setup of the devices, and programming of the IVSPs and nurses favoured the usability of the pneumatic over the peristaltic IVSP.
For the aim 1, safety measure of medication error (aim 1a), the majority of the medication errors occurred during secondary infusion, a problem that has been described in previous research.21–23 Because of space and height limitations, especially in the critical care environment, the manufacturer required setups for the peristaltic IVSP are difficult, if not impossible, to achieve during actual clinical use. This is an example of a medical device that was not designed for the environment in which it is being used, creating a safety hazard for both patients and nurses. As shown in table 2, the types of medication errors that occurred represent frequently occurring, occult intravenous medication administration errors that should be of concern. While individual patient harm for each instance is difficult to assess, in total, these errors may have a role in antibiotic resistance. Incomplete dosing of antibiotics not only decreases the therapeutic benefit to the patient but also contributes to antibiotic resistance due to the resultant subtherapeutic levels which foster bacterial mutation.29
For the aim 1, safety measure of interruptions (aim 1b), there were more interruptions during programming the peristaltic versus pneumatic, which is due, at least in part, to the increased time it takes for programming the peristaltic, a finding that has been reported in the previous research.30 Data also support that the number of and severity of medication errors increase with interruption frequency, making this an important finding with regards to patient safety.14 17
The aim 1 safety measure of compliance with using the drug library and DERS (aim 1c) was significantly higher with the pneumatic versus peristaltic. While both IVSPs have options to override the DERS and use ‘basic infusion’ options, the pneumatic IVSP requires the same amount of effort to programme basic infusions as it does to programme within the DERS, reducing the likelihood of using this option to bypass programming using the DERS. The availability of the pneumatic touchscreen also simplifies the programming experience.30
For the aim 2, usability measure of compliance with manufacturer required setup (aim 2a), the 0% compliance for the peristaltic IVSP should be of concern to all practicing clinicians in the acute care setting. Given what is known about the need for correct system setup to facilitate accurate flow, non-compliance with IVSP location relative to the patient (55.7%), location of the primary fluid bag (100%) and location of the secondary relative to the primary (37.2%) means that a significant number of undetected under-infusions occurred in the observed medication administrations. This has also been reported in the previous research.21–23
For the aim 2, usability measure of programming workload (aim 2b), the NASA-TLX indicated that the pneumatic was associated with a significantly lower perceived workload versus peristaltic IVSP. Combined with the significant reductions in programming time for the pneumatic versus the peristaltic and the significantly increased ease-of use for the pneumatic, these findings are particularly strong, given that the nurses in this study were experienced peristaltic users.
Limitations
The purpose of this study was to compare the two IVSP types during actual clinical use. However, since the pneumatic IVSP had never been used in a clinical setting, the investigators limited the clinical study sites to two small units with consistent staff and patient types. This limited the number of observations and the overall generalisability of the findings. We recommend that more studies of this kind be conducted with wider variety of patient types, with more nurses and on busier clinical units such as medical-surgical and critical care. The study was conducted in a single site, which may not represent practice in all acute care settings. Due to limited functionality of the digital logs and portal for data in the peristaltic IVSP, metrics for DERS compliance and alarms/alerts were extrapolated from larger groups of like units where the peristaltic was used, which limited our ability for a more direct comparison of DERS compliance.
Conclusion
This study is the first IVSP usability comparison to be conducted in an actual clinical setting and, thus, represents important, real-world clinical evidence. Results found that both the safety and usability measures used in this study support overall improved clinical use with the pneumatic versus the peristaltic IVSP, a finding that is important for both clinical practice and patient safety. Continuous innovation in IVSPs to develop technical solutions for as many usability issues as possible is an important part of improving safety of intravenous medication administration. Both the use of an intuitive touchscreen and the development of technology not requiring a head-height differential for accurate fluid flow are features that improve IVSP usability. With the high level of demand for clinicians at the point of care, manufacturers have the responsibility to continually innovate to improve IVSP usability in this very important area of patient safety.
Data availability statement
All data relevant to the study are included in the article.
Ethics statements
Patient consent for publication
Ethics approval
This study involves human participants and was approved by The Orlando Health Number 1 IRB 20.107.07 Participants gave informed consent to participate in the study before taking part.
References
Footnotes
Correction notice This article has been corrected since it was published. Surname of Dr. Giuliano has been corrected.
Contributors DP conceptualised the manuscript, wrote the manuscript and supplemental figures and tables and DP coordinated all other author feedback and revisions for the submission. KG led the writing of the background, created the conceptual framework, and contributed significantly to drafting of various aspects of the manuscript, content development and editing. AM contributed the tables for the manuscript and performed contribution to data presentation and editing. DP and KG are guarantor.
Funding Funding for the study was provided by an industry research grant from Ivenix, Inc. through their investigator-initiated grant program.
Competing interests In addition to the funding statement described above, DP has received industry-funded grants for related studies for investigator-led research from ICU Medical and Ivenix. KG has received study support and has performed consulting services for Ivenix and ICU Medical. AM has received salary support from industry grants from Ivenix and from ICU Medical.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.