Article Text
Abstract
Background Surgical technology has led to significant improvements in patient outcomes. However, failures in equipment and technology are implicated in surgical errors and adverse events. We aim to determine the proportion and characteristics of equipment-related error in the operating room (OR) to further improve quality of care.
Methods A systematic review of the published literature yielded 19 362 search results relating to errors and adverse events occurring in the OR, from which 124 quantitative error studies were selected for full-text review and 28 were finally selected.
Results Median total errors per procedure in independently-observed prospective studies were 15.5, interquartile range (IQR) 2.0–17.8. Failures of equipment/technology accounted for a median 23.5% (IQR 15.0%–34.1%) of total error. The median number of equipment problems per procedure was 0.9 (IQR 0.3–3.6). From eight studies, subdivision of equipment failures was possible into: equipment availability (37.3%), configuration and settings (43.4%) and direct malfunctioning (33.5%). Observed error rates varied widely with study design and with type of operation: those with a greater burden of technology/equipment tended to show higher equipment-related error rates. Checklists (or similar interventions) reduced equipment error by mean 48.6% (and 60.7% in three studies using specific equipment checklists).
Conclusions Equipment-related failures form a substantial proportion of all error occurring in the OR. Those procedures that rely more heavily on technology may bear a higher proportion of equipment-related error. There is clear benefit in the use of preoperative checklist-based systems. We propose the adoption of an equipment check, which may be incorporated into the current WHO checklist.
- Safety culture
- Checklists
- Patient safety
- Healthcare quality improvement
- Surgery
Statistics from Altmetric.com
Introduction
There can be no doubt that the development of technology in the operating room (OR) has led to significant improvements in outcome, reducing postoperative patient morbidity and mortality. However, the increasing use of technology in all surgical specialties may also increase the complexity of the surgical process and may represent an increasing propensity to error from equipment failures.1 In order to achieve the highest quality of care for patients, we need to strive to develop technology for improving patient outcomes and develop systems and training pathways that allow effective, safe use of specialised equipment.
Medical errors affect up to 16% of patients admitted to hospital and account for significant additional morbidity and mortality.2 ,3 Approximately one half of these adverse events are attributable to surgical procedures.4 ,5 Failures in equipment and technology are a major implicated factor in surgical errors and adverse events.
Identification and minimisation of equipment-related failure is therefore an important aspect in improving patient safety and the efficiency of the surgical procedure. Moreover, equipment and devices (as opposed to other types of errors, eg, communication, technical, etc) are potentially the most conducive to objective checking and error reduction by means of checklist systems, which have been shown to be effective, not just in preventing error preoperatively, but in highlighting equipment problems, which would otherwise persist in contributing to error from one operation to the next.6
The exact contribution of equipment to overall error is unclear: our recent study of 66 complex vascular and endovascular procedures revealed this to be as high as 24%.7 The role of equipment-related error in surgery, and the effect of increasing technological complexity remains to be characterised.
The specific objectives of this systematic review are to determine the following:
-
the proportion of overall surgical error in the OR occurring due to equipment and technology
-
the characteristics of equipment failures recorded
-
procedures that are associated with a higher proportion of equipment failures
-
other factors affecting reported equipment error rates.
Methods
A comprehensive, structured review of published articles was performed. The overall strategy included a search for, and selection of, studies which reported data on error or adverse events occurring in the OR, including the contribution of equipment or technological errors. Initial search results were sequentially filtered using progressively introduced specific inclusion and exclusion criteria. The papers finally selected were analysed to determine the role of equipment errors in safety failures in the OR.
Data sources and search terms
The electronic databases: MEDLINE (1965, September 2012), EMBASE (OVID) (September 2012), and PsycINFO (OVID) (1965–week 4, September 2012), HMIC (OVID) (1979–September 2012) were searched for studies relating to error or adverse events in the operating theatre. Additional searches were also carried out on Google Scholar and Cochrane Database of Systematic Reviews. We devised the search strategy by initially compiling keywords from key papers and broad literature searches on these electronic databases. Search terms were refined through an iterative process of reviewing outcomes of preliminary keywords searched in the databases. Two groups of terms were used as listed below. MeSH (Medical Subject Heading) terms were used to broaden the scope of the search. The Boolean terms ‘AND’, ‘OR’ were used to combine the terms of the first group (1) with those of the second group (2). Terms relating to equipment malfunction or technology were intentionally omitted at this stage, as this narrowed the scope of the search too much.
Group 1:
“Surgical procedures”
“Operating rooms”[Mesh]
Group 2:
“Risk assessment”
“Risk management”[Mesh]
“Safety”[Mesh]
“Medical errors”
“Malpractice”[Mesh]
“Insurance claim review”[Mesh]
“Systems analysis”[Mesh]
“Task performance and analysis”[Mesh]
Study selection
From the total search results, studies were subsequently selected through a stage-by-stage process (summarised in figure 1). First, publications not in the English language and not relating to humans were excluded. Secondly, publications not describing a research study (namely editorials, reviews, letters, case reports and case series) were excluded and duplicate publications removed.
Abstracts of the remaining 2119 texts were screened by at least one author (RW, CB) to select reports for full-text review. Using a standardised quality assessment tool and prespecified inclusion and exclusion criteria, two authors then independently assessed each of the remaining 124 articles for inclusion in the study, resolving disagreements by consensus of input from a third author (CR).
Quality assessment
There being a lack of consensus criteria applicable to such studies, we modified a quality assessment tool previously used in our department, which was synthesised based upon existing general recommendations on quantitative studies (eg, MOOSE, QUORUM).8–10 In brief, this assesses each publication using a number of predefined criteria (each scored on a 3-point scale: 0=criterion not met; 1=partially met; 2=fully met) (see online supplementary appendix 1). Two authors (CR, CB) not involved in subsequent data analysis undertook independent quality assessment during full-text shortlisting, prior to deciding for or against final inclusion of a study. To be considered for inclusion, publications had to meet a minimum mean quality score of 1.0, the score whereby all applicable quality criteria were at least partially met (see online supplementary appendix 1).
Inclusion criteria
To be included in the review, studies had to meet the following criteria:
-
systematically and quantitatively assess errors and/or adverse events occurring during, or as a result of, surgical procedures
-
include analysis of equipment failures as part of the study
-
allow quantification of the contribution of equipment failures to total error.
‘Equipment’ included any medical device or apparatus normally required during the course of an operation (cognitive artefacts, such as medical records were not included); ‘equipment failure’ included lack of availability or incorrect function/settings recorded as causing disruption to the normal process of the operation (including disruptions to workflow): this was kept broad to allow for capture of latent and minor errors.
Exclusion criteria
-
Study design: (a) qualitative studies or (b) quantitative studies which did not quantify the contribution of equipment/technological failure.
-
Study context: outside the scope of the surgical procedure/operating room or not specifically including the surgical procedure. (Some studies whose scope extended beyond the operating room alone but included relevant data specifically attributable to the surgical procedure/operating room, were considered).
Data extraction and analysis
A standardised spreadsheet was designed to assimilate summary data from each study. Data sought and recorded for each study included: study design, year, confirmation of inclusion criteria met, number of subjects, number and types of operative procedure studied, total error and its categories, equipment error and its subtypes. In studies describing interventions to reduce error, and where error severity was noted (major vs minor), this distinction was preserved.
Further analysis allowed mean, range, median and interquartile range (IQR) for all applicable studies to be calculated across all studies for three main indices:
-
total error rate per operation (total errors divided by number of operations)
-
the percentage of total error contributed by equipment error
-
equipment error rate per operation (all equipment errors divided by number of operations).
Finally we analysed the above indices in relation to error severity (major vs minor error), operation type and checklists/interventions.
Data from studies of clearly different design (eg, malpractice claim reviews, retrospective and prospectively conducted analyses) were analysed separately to minimise the effect of methodological bias, as we anticipated that overall study design may influence reported error rates (see following section). Furthermore, median values were reported in preference to mean values where there was significant skew owing to outliers.
Results
Study selection
From the initial 19 362 citations yielded, 124 were selected for full-text review (figure 1). From these, 27 publications met the inclusion criteria. One publication described two distinct studies, therefore, a total of 28 studies were counted for the purposes of this review. All 28 studies met the minimum quality score of 1.0 (mean score 1.5). All were systematic, quantitative analyses of error and followed one of three main designs:
Error rates by study type
1. Malpractice claims
These studies highlight significant errors that lead to litigation. The four studies collectively reviewed 1285 claims, in which a median 15.5% and IQR 12.5%–17.5% of cases were contributed by equipment/technological failures. They did not record data in sufficient detail to calculate error rate or the specifics of the equipment errors.
2. Retrospective studies
Three retrospective studies involved either reviewing perioperative incident reports32 or interview questionnaires with surgeons, either in person30 or in written form.33 Again, such studies of error do not have the data to calculate error rate; however, of the incidents recorded, a similar proportion was attributable to equipment/technological failures: median 15.0% (IQR 12.2–16.0%).
3. Prospective studies
Prospective studies comprised the largest group (21 studies) and were further divisible into two types:
-
Self-reported: those that were reported by the surgeon or surgical team immediately after each operation (seven studies).17 ,21 ,24–27 ,29
-
Independent observer: those that were recorded according to a preagreed systematic protocol by an independent assessor (ie, someone not directly involved in the operation) (14 studies).7 ,11–20 22–23 ,28
These studies provided more detailed information, which was elicited in the following categories:
A. Total error rate (total errors per operation)
Reported total error rates varied widely between studies (table 1). Across all prospective studies, the median-reported total error per operation was 2.42 (IQR 1.2–16.5); mean 8.6 (range 0.03–32.8). (This figure takes into account only the preintervention data in the cases where the effect of an intervention was studied). Independent observer-assessed prospective studies reported higher error rates (median 15.5 IQR 2.0–17.8) compared with self-reported assessments of adverse events (median 0.4, IQR 0.17–0.9).
B. Equipment-related error
Across all prospective studies, the median proportion of total error due to equipment failure was 23.5% (IQR 15.0–34.1%), and this was similar between independently assessed and self-reported studies (table 2). The median number of equipment problems per procedure was 0.9 (IQR 0.3–3.6) (table 2). There was strong correlation between equipment-related error rate and total error rate per operation (R2=0.76).
C. Subtypes of equipment error
From eight studies,7 ,13–15 ,22 ,25 ,32 sufficient information was available to subdivide the equipment failures into three groups (partly characterised in previous studies7 ,14) as follows (figure 2):
-
Availability: lack of availability of equipment when required, accounting for 37.7% (range 19.4–47.6%) of all equipment failures.
-
Configuration and settings: problems with the configuration and settings of the equipment, preventing normal use, seen in 43.4% (range 22.7–73.1%).
-
Direct malfunctioning/failure: accounting for 33.5% (range 15.1–60.0%).
Specific examples of each subtype of equipment error, based on type of operation, are shown in online supplementary appendix 2.
Other factors affecting reported error rate
Major versus minor error
Four studies12 ,14 ,15 ,28 characterised error severity as ‘major’ (broadly defined as a major disruption to the operation or with potential adverse outcome to the patient) or ‘minor’ (all other types of error, causing any disruption to the normal flow of the procedure). From these studies, 21.2% of all error was classified as ‘major’. Equipment problems were implicated in 20.8% of all major errors; this compares with 8% and 13% for communication and technical failures, respectively.
Type of operation
Total and equipment-related error changed with type of operation: the pooled results from the six studies of cardiac operations12 ,14 ,18 ,19 ,23 ,26 showed higher total and equipment problems compared with the five studies of miscellaneous general surgical procedures11 ,13 ,16 ,20 ,22 (for equipment error, cardiac surgery: 2.99 (range 1.7–4.1) vs general surgery: 0.79 (range 0.7–0.86); for total error, cardiac surgery: 18.45 (9.5–32.8) vs general surgery: 2.0 (0–5.9) (figure 3)).
Two studies used the same methodology to compare operations of different types (and technological complexity).7 ,28 Each showed differences in the rate of total error and equipment-related/technology-related error between type of procedure (table 3).
Effect of checklist or other intervention
In six studies, the effect of a safety intervention on error rate was measured.13 ,16 ,18–20 ,22 Interventions included one or more of the following: preoperative checklist, perioperative briefing tool or staff training programme. Total error rate per procedure was observed to decrease in three out of four studies following implementation of the intervention with a mean reduction of 38.5% (range 20–64.2%). Equipment problem rate per procedure reduced by a mean value of 48.6% (range: 7.4–83.7%) after introduction of interventions. In three studies where the intervention was a checklist specifically including an equipment check, a mean reduction in equipment error of 60.7% (range 34.2–84.7%) was observed.13 ,16 ,22 The combined data did not reveal statistically significant changes in error rates due to the small number of studies.
Discussion
This is the first systematic review specifically studying the contribution of equipment and technology failures to overall error in the operating theatre. Accepting the degree of heterogeneity in the processes and methods of studying error, we believe that such a review is useful and particularly pertinent given the increasing introduction of technology in the operating theatre at the current time. It also provides a useful opportunity to investigate an area that would potentially respond well to error-reduction measures.
Error rates and equipment
Prospective studies
The prospective studies offer the most systematic and comprehensive method of capturing all errors that occur during the course of an operation, with independent-assessor studies representing the most comprehensive and least biased method.
These studies also show variability in reported error rates, which partly reflects the heterogeneity of the current body of evidence both in terms of procedures studied and methodology used to collate the various types of error. It is difficult to differentiate between the relative effects of type of procedure versus methodology on reported error rates. It is likely that study design (prospective vs retrospective vs malpractice) has an impact on reported error rates; furthermore, variation in error-recording scheme and definitions of errors may also have an influence (eg, two contrasting examples can be seen in the study of laparoscopic cholecystectomies,13 where error criteria are very specific and itemised, compared with Henrickson's study of flow disruptions in cardiac surgery18 where very broad categories are used to capture error (‘Procedure related’, ‘Equipment related’, ‘Patient related’ and ‘Communication’ errors).
On the simplest level, this is evident in the increasing trend in mean error rates from malpractice and retrospective studies to self-reported prospective studies to prospective independently observed studies. Looking at the latter group, the range of error rates overlapped between those prospective studies observed by medical (n=6) vs non-medically trained (n=7) observers and the single study which used both showed high interobserver agreement.
Nevertheless, it is clear from these studies that equipment problems contribute a substantial proportion of all error. Furthermore, it seems that when severity of errors is considered, in some studies a higher proportion of equipment errors may be considered ‘major’ errors than for other categories of error such as communication and technical errors.
Equipment error was observed to positively correlate with overall error rate. This may reflect a common cause intrinsic to the overall system, or reflect the different methods of error capture techniques, and may also be related to operation length.
Malpractice and retrospective studies
Studies of malpractice claims traditionally provided the first analyses of medical error in general, though the four studies meeting our inclusion criteria were relatively recent. These four studies have been instructive in two respects: in demonstrating a significant role for equipment failures in error and in demonstrating its specific importance with regard to medical litigation, one important consequence of medical error. Retrospective studies, similarly, give a specific insight into the role of equipment failures with regard to significant errors remembered some period after the event, or as recorded in incident reports.
Both study designs are, however, limited in terms of completeness, for example, they do not cover every operation over a given period, thereby precluding error rate calculation or any detailed quantitative analysis. It is inevitable that with these methods there may be a tendency to under-report all error, by focussing on the causes of adverse events rather than studying errors that occur without appreciable patient harm when taken in isolation—latent errors.15 There may also be a bias away from errors that are not directly patient-related (such as equipment error). This may explain the significantly lower proportion of equipment-related error reported in these studies (approx 15%) compared with the prospective group (21–23%).
Technological complexity and error-rates
On the basis of our preliminary observations, it seems that as the complexity of technology increases, the propensity to equipment error may also increase. Our analysis suggests that equipment error may be higher in operations where more complex technology is used, even though total error may not. Certainly, comparing all prospective independently observed studies of cardiac surgical procedures with those of general surgical procedures, the observed total and equipment-related error rates are universally higher in the former. Indeed, the error rates from two individual studies of other procedures known to have complex technological involvement—endovascular operations7 and deep brain stimulation (DBS)28—also show equipment-related error rates comparable with the cardiac surgical group (4.1 and 2.5 mean equipment errors per operation, respectively) (figure 3).
In our own study comparing complex open vascular versus endovascular procedures, we observed a similar pattern, with a higher proportion of equipment-related error in the technologically more complex endovascular group (52% vs 11%).7 This difference in error remains when one considers the rate of error per hour (controlling for the length of procedure): 3.0/h in the open vascular surgery group and 9.6/h in the endovascular group.7 The only other study of procedures of different technological complexity was that by Catchpole et al,15 which recorded errors in 24 paediatric cardiac operations and 18 general orthopaedic operations (table 3), although comparison of error between procedures was not the purpose of this study. Here, error rates were higher in the latter group, probably explicable by the different error-marking schema used for each group. The key strength of this study, however, was in successfully applying standardised higher-level error capture and systems-improvement methods to two contrasting operative environments.
Finally, it must be borne in mind that the evaluation of error rate between operations of different technological complexity rests mainly upon interstudy comparisons (each with different schema for marking error). Furthermore, there is a knowledge gap in terms of error rate controlled for operative duration (error rate per hour).
Checklists
Preoperative and perioperative checklists have already been established as key tools to improve safe surgery.6 Of the eight studies which examine the subtypes of equipment error, it is significant that 70% of equipment errors are attributable to equipment availability (32%) and configuration (38%). This would suggest that a large proportion of equipment-related error might be avoidable by adequate preoperative checks. Clearly, there are factors, which are non-predictable, such as direct intraoperative malfunctioning of equipment or missing items from prepacked operating sets. However, even for the latter, such a system would provide a ‘front-line’ recognition process, which could be fed back to and complement existing checks at the manufacturing/central sterilisation stage. It may also be useful to highlight potential staff training needs (in the case of equipment configuration errors). The concept of equipment error prevention is supported by the studies that include a safety intervention, which show significant improvements (equipment error of up to 84% and all error rate of up to 64%) by the introduction of a preoperative checklist/briefing tool.12 ,19 ,20 This is further appreciable when one considers the specific examples of equipment problems quoted (see online supplementary appendix 2), which are broadly shared, irrespective of the type or specialisation of the operation.
Limitations and Further work
The studies differed too much in their details (design, context, measures) to undertake valid meta-analysis. Few studies exclusively attend to the question of equipment problems, and there is a significant knowledge gap in terms of procedure-specific equipment error rates. Similarly, studies using the same methodological framework to study error rates in operations of varying technological complexity, and recording error rate controlled for operative duration, are limited and are clearly an area for further research. This analysis also highlights some of the difficulties with studying error in the operating theatre, mainly attributable to the variety of methodologies used and the need for subgroup-based analysis to take into account these variables when making quantitative estimates. The patently lower rate of total error reported in retrospective and self-reported studies may suggest a bias toward under-reporting in these studies (with a bias toward major vs minor error) and that independently assessed prospective studies by dedicated observers probably provide the ‘gold standard’ analyses of error in the surgical environment. Finally, few studies quantified error severity or categorised into ‘major’/‘minor’ types, meaning our analysis of this area was necessarily limited.
Recommendations
In spite of the heterogeneity of studies, operation types and numerous other factors which influence reported error rates, it is clear that equipment problems form a significant proportion of total error occurring in the OR, and that a large proportion of this error is preventable by robust preoperative safety systems. We would therefore recommend the development of a generic checklist system that addresses each of the main categories of equipment problems—availability, configuration and settings, direct malfunctioning/failure—that have been identified in this and previous analyses, and may also be specifically adapted to reflect the particular technology of any procedure. Such a checklist may be carried out relatively seamlessly as an addendum to the current preoperative checks of WHO Surgical Safety Checklist (as some studies have already demonstrated in priniciple12 ,19 ,20 ,37 ,38). The feasibility and efficacy of such a checklist in the context of routine surgical practice would need robust quantitative evaluation. Clearly there is also work that needs to be done training teams to work with technology, and developing user-friendly systems, to allow the integration of technology into the operating theatre as a whole, without increasing error rates: such a goal would clearly require a systems approach, taking into account human factors and their interaction with the systems introduced.39
Conclusion
Equipment-related failures form a substantial proportion of the cause of all error recorded in studies of operating room safety failures, but with widely variable values reported. Those procedures that rely more heavily on technology may bear a higher proportion of equipment-related error. There may be great scope to improve the results of procedures over and above that already achieved from the introduction of new technologies by paying due attention to potential errors related to equipment.
There is clear benefit in the use of preoperative checklist-based systems, by which a large proportion of equipment-related error and overall error can be reduced. We propose, among other training and system designs, the evaluation of a generic equipment check which may be incorporated into the current WHO checklist. Further studies are required to quantify the categories of equipment-related error specific to key groups of procedures as well as the effect of checklists.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Files in this Data Supplement:
- Data supplement 1 - Online appendix 1
- Data supplement 2 - Online appendix 2
Footnotes
-
Contributors RW, CB and NC—conceived idea and study strategy; RW, CB, CR and RL—literature search, quality scoring, and shortlisting full text; RW and CR—data analysis; RW, NC, MH, KM, AD, CV and CB—synthesis of summary data; writing, critical appraisal and revision of manuscript (multiple iterations).
-
Funding This study was part funded by the Imperial College Healthcare Trust and the NIHR through the Comprehensive Biomedical Research Centre. The Clinical Safety Research Unit is affiliated with the Centre for Patient Safety and Service Quality at Imperial College Healthcare NHS Trust, which is funded by the National Institute of Health Research.
-
Competing interests None.
-
Provenance and peer review Not commissioned; externally peer reviewed.
-
Data sharing statement Additional data available on request from the corresponding author.