Article Text
PDF
Abstract
Acute abdominal pain is a common reason for emergency presentation to hospital. Despite recent medical advances in diagnostics, overall clinical decision-making in the assessment of patients with undifferentiated acute abdominal pain remains poor, with initial clinical diagnostic accuracy being 45–50%. Computer-aided decision support (CADS) systems were widely tested in this arena during the 1970s and 1980s with results that were generally favourable. Inception into routine clinical practice was hampered largely by the size and speed of the hardware. Computer systems and literacy are now vastly superior and the potential benefit of CADS deserves investigation. An extensive literature search was undertaken to find articles that directly compared the clinical diagnostic accuracy prospectively of medical staff in the diagnosis of acute abdominal pain before and after the institution of a CADS programme. Included articles underwent meta-analysis with a random-effects model. Ten studies underwent meta-analysis that demonstrated an overall mean percentage improvement in clinical diagnostic accuracy of 17.25% with the use of CADS systems. There is a role for CADS in the initial evaluation of acute abdominal pain, which very often takes place in the emergency department setting.
- Abdomen—non-trauma
- assessment
- cardiac care
- clinical assessment
- clinical care
- diagnosis
- major incidents
- mental health
Statistics from Altmetric.com
- Abdomen—non-trauma
- assessment
- cardiac care
- clinical assessment
- clinical care
- diagnosis
- major incidents
- mental health
Patients with acute abdominal pain commonly present to the emergency department (ED) and represent approximately 18% of surgical admissions in the UK.1 2 Despite recent medical advances in diagnostics, overall clinical decision-making in the assessment of patients with undifferentiated acute abdominal pain remains poor with initial clinical diagnostic accuracy being 45–50%.1 Studies have shown that the use of structured data collection forms and computer-aided feedback may improve this figure by approximately 10% each.3 Furthermore, the use of such systems has demonstrated other benefits including reductions in hospital admission rates3 and in the number of negative appendicectomies and laparotomies performed.2 Although the cause for this was difficult to understand,4–8 the principal reason appears to be a more accurate diagnosis of non-specific abdominal pain.3
There is clearly room for clinicians to improve their performance in this area. Computer-aided decision support (CADS) systems were widely tested in the arena of acute abdominal pain during the 1970s and 1980s. Although results were generally favourable, the hardware was cumbersome and data entry slow such that uptake of such systems into routine clinical practice has been poor. Hardware, software systems and most importantly computer literacy are now vastly superior and the potential benefits of CADS deserves investigation and a review of the literature in this field.
Objective
To perform a systematic review of the literature in order to answer the following three-part question: in adult patients with acute abdominal pain, does the use of computer aided clinical decision support improve clinical diagnostic accuracy?
Methods
An extensive search of the Cochrane, Medline (1951–April 2007), Embase (1974–April 2007) and CINHAL (1982–April 2007) databases was undertaken (search strategy in appendix 1, available online). Expert authors were sought and contacted, reference lists scrutinised and best evidence resources searched. An internet search was also made by means of the Google interface. Duplicate articles were removed and abstracts studied for relevance to the study question. Those selected underwent detailed critical appraisal and only those trials that prospectively directly compared the clinical diagnostic accuracy of medical staff in the diagnosis of undifferentiated acute abdominal pain before and after the institution of a CADS programme were selected to undergo meta-analysis.
For both arms of each trial the number of patients correctly diagnosed is assumed to follow a binomial distribution and thus a 95% CI can be calculated for each estimate of the correct diagnosis rate, and mean changes and standard errors were extracted from the published material.
To reflect the differences that are likely to be apparent in the trials selected due to different hospitals, clinicians, patient populations and geographical areas a random-effects meta-analysis was used. A forest plot and analysis table were produced using STATA 9.0 software and a test for heterogeneity was included.
Results
In all 20 prospective studies2–4 9–25 were selected that were pertinent to the question posed (figure 1). There were no meta-analyses and no systematic reviews. There was one randomised ontrolled trial,4 but this only partly answered the question. The other 19 studies were prospective comparative studies. Ten studies were excluded from the meta-analysis due to: a lack of baseline data11; subset data published from a bigger trial16–18 22 25; missing post-CADS data21; long-term results without figures2; a retrospectively validated database4 and one trial since the comparing database was small and untested.24 Ten studies were entered into a random effects meta-analysis (figure 2).
Flow diagram.
Forest plot of included studies.
Discussion
Eight of the 10 trials show a clinically significant improvement due to the introduction of CADS, the exceptions being the studies reported by Wilson et al15 and Kirkeby and Risø.20
Both statistically and clinically, there is significant heterogeneity among the trials and therefore the combined result should be regarded with some caution. This is expressed by the I2 value of 91.5%—that is a great majority of the variation occurs between trials rather than within them. This may be due to variations in the patient populations or in the administration of the CADS support at different sites. For example, a different grade of physician may be using the tool. It would be reasonable to expect little improvement if the physician had considerable experience and that maximum benefit would accrue for the least experienced physicians.
Overall, the results of this meta-analysis demonstrate a mean percentage improvement in diagnostic accuracy of 17.25% (95% CI 16.44% to 18.07%), although the analysis is largely dominated by one large multinational trial.9 Together, they suggest a role for CADS in the initial evaluation of acute abdominal pain, which very often takes place in the ED setting.
Despite the advent of sophisticated imaging techniques, the assessment of the acute abdomen is still very much a clinical domain. This meta-analysis pools all the evidence on this subject and demonstrates an improvement in clinical diagnostic accuracy when a CADS system is combined with the clinical acumen of front door clinicians in the assessment of undifferentiated acute abdominal pain. Correct initial diagnosis leads to more timely and appropriate treatment, referral, observation or discharge of patients.
Computerised decision support in this field was pioneered by a group from Leeds led by de Dombal and colleagues.26 Their initial database was compiled from 600 structured case histories26 of patients presenting with acute abdominal pain. New patient indicants were recorded on structured data collection forms, coded and then entered into a computer system.27 The database then used a system of Bayesian analysis and comparison with the structured case histories to generate diagnostic probabilities with feedback given to clinicians.
The initial study13 demonstrated spectacular results, with CADS producing a clinical diagnostic accuracy of 91.8%, higher even than the senior clinician (79.6%). These results were maintained through a second study period10 only to decrease again when the trial finished. A smaller desktop system was developed28 and tested11 with positive results. The Leeds system was successfully exported to West Lothian, Scotland,29 although improvements in initial clinical diagnostic accuracy (55–77%), early discharge and negative laparotomy must be largely attributed to structured data collection in the environment of audit because any computer feedback was much delayed.14
This, and other work also drew attention to the poor initial clinical diagnostic accuracy (39–55%) of ED clinicians in the assessment of acute abdominal pain.10 11 13 14 Patients presenting to the ED are assessed by junior staff earlier in the timescale of their disease when clinical signs may be less evident and case notes not available. Wilson et al15 set out to test the Leeds system on an unselected prospective series of 1196 patients attending an urban ED with acute abdominal pain. The diagnostic accuracy of the clinician alone was 65.6%, better than with CADS (65%). This largely reflects the use of a database designed purely for general surgical conditions on a population of much broader case mix presenting to the ED.
The Leeds system was improved to include gynaecological and urological disorders,30 and now had a database of over 6000 cases,30 when a large prospective trial in eight different centres across England was undertaken to see if results were generalisable nationally.12 Initial clinical diagnostic accuracy improved from 46% to 65% and, at those centres where patients with acute abdominal pain presented to the ED (baseline n=2871, trial n=9150), admission rates fell from 55.8% to 47.4% (p<0.001) and of those of non-specific abdominal pain patients from 40.9% to 26.3%. Results were not as dramatic as initial studies10 13 but probably represented what could realistically be achieved in the NHS. These systems were incorporated into routine practice in some centres, with improvements sustained over many years,2 21 22 and further work was performed on an international European population demonstrating improvement in initial clinical diagnostic accuracy from 48% to 66%.9
Evidence against the use of CADS systems in acute abdominal pain
Sutton had been part of the Scottish group2 and had levelled several criticisms about CADS in general5 and about computer accuracy in particular. He published the only large study suggesting that the use of a CADS system conferred no benefit over clinicians.4 The study has a number of problems and did not meet inclusion criteria for entry into our analysis. The DIAG database was not properly described, nor was it prospectively or externally validated. The use of computer and structured data collection forms is very poor in the prospective arm of the study and comparison is made between DIAG and a database (CAD-A) that does not recognise gynaecological and urological causes of acute abdominal pain. Furthermore, the numbers are small and cases in which no definite diagnosis is made are excluded. Sutton4 concluded that the computer probably adds nothing to the accuracy of the diagnosis in acute abdominal pain and suggested future efforts should be directed to low-cost solutions such as detailed history and examination templates and audit.
A study from rural Norway,20 using a slightly modified version of the Leeds database with Norwegian patients found an initial clinical diagnostic accuracy of 65% (50/77) compared with the 53% achieved with the CADS system. Although a small study, the negative results may reflect a different referral pattern and the fact that the database failed to classify 18% of cases compared with 3% in the 1972 study by de Dombal et al.13
An American group24 developed an algorithm and used a microcomputer to encode knowledge-based information about different signs and symptoms in acute abdominal pain representing the relative importance that an expert would assign to them. They thus computed a normalised score for each disease based on the sum of individual scores and also incorporated rule-based logic. Clinician diagnostic accuracy was 66% and the computer's accurary was 60%, although the test population was small.
Some centres involved in the large European study9 showed little improvement in clinical diagnostic accuracy with the use of CADS.25 This seemed principally due to the fact that the doctor initially assessing the patient in these centres was a senior registrar or consultant and was at odds with the overall multicentre results.9
Effect of different components of CADS systems
Sutton5 estimated that half the improvements generated by CADS are due to structured data collection forms. This opinion has been echoed by others.14 19 31 However, when the computer was not available the negative laparotomy and perforated appendix rates reverted towards baseline rates even though the forms were still being used.2 22 Despite careful data collection, the accuracy of any CADS system is crucially dependent on the consistent use of definitions (eg, guarding) and interpretation of clinical signs (eg, rebound). The importance of feedback was first noted by Wilson et al11 and confirmed by de Dombal et al.23 Adams et al12 found significant contributions of approximately equal size by forms, computer and feedback. In an effort to combat difficulties with computer inception in this field32 33 the computer was used as an educational tool with good effect.14 23 Many authors were mystified as to why there should be such dramatic improvements in clinical performance (length of stay, admission rates, negative laparotomy and negative appendicectomy).4–8 Wellwood et al3 found that most of the benefits were almost entirely explained by a more accurate diagnosis of non-specific abdominal pain.
Present situation
It is clear that there is much room for the clinician to improve in this field. Sutton5 has stated that acute abdominal pain is a good test arena for CADS systems: it is clinically important and has a finite number of possible diagnoses.
The results of subsequent studies,2–4 9 12 14–18 21 22 including two large multicentre studies,9 12 suggest that the expected initial clinical diagnostic accuracy with computer assistance is of the order of 60–70% and that it is generalisable across different sites in the UK and Europe. However, there is a learning period to its use11 14 and the specificity of computer diagnosis may be poor.3 Other benefits of CADS systems include reduced rates of perforated appendix2 9–12 16 18 22 and negative laparotomy,2 9–12 14 16–18 22 which should result in benefits financially and in terms of bed occupancy. These systems can be incorporated into routine practice and show sustained benefits for many years.2 21 22
The principal reasons for failure to adopt systems of CADS into routine practice in the 1970s and 1980s included the practicalities of extra work32 and an inherent reluctance to incorporate computers into clinical decision-making.33 Data had to be recorded twice (on to forms and then into the computer), the hardware was bulky and non-portable and the software slow. Further work in this field largely stalled with the sad death of Professor Tim de Dombal in 1995.
Doctors are now more computer literate and most accept that computers play an important role in the provision and analysis of data in order to allow medical staff to make the best decisions. Diagnostic software such as AAP Help1 can be run on a palm top computer or from remote sites via the internet, and paper print-outs of data can be generated for the notes to avoid duplication of work, and it is feasible that it could become part of routine clinical practice.
Databases are now very powerful and cover the full spectrum of diagnoses of acute abdominal pain presenting to the ED,1 although they are still dependent on consistent recording of data and interpretation of clinical signs. Individual clinicians may well be able to look at the computer-generated probabilities with and without a specific clinical sign (eg, rebound) if they are unsure.
The definitive trial as to whether CADS has a clear and definitive role in the diagnosis of acute abdominal pain has not yet been performed. However, the results of this meta-analysis and the weight of the prospective evidence in thousands of patients demonstrates that its use in this clinical situation results in reduced inhospital admissions, length of stay, negative appendicectomy rates and economic benefits to the hospital.
Limitations
The literature search strategy was extensive and included studies in all languages. However, no randomised controlled trials fully addressed the clinical question posed. Indeed, a randomised controlled trial in this arena would be fraught with difficulties in its inception. The evidence studied was therefore not of the highest standard but certainly was pragmatic and involved trials on a large number of patients. The suitability of studies to be entered into the analysis was only determined by one author. The inclusion criteria were tight, but the review may have been stronger if multiple authors had assessed the articles for inclusion. The results showed significant heterogeneity within the trials, which mandates caution in the interpretation of the results. This had been anticipated and therefore a random effects model of meta-analysis was used appropriately.
Conclusions
CADS shows potential benefits in the management of patients with acute abdominal pain attending the ED. The previous difficulties with its inception into routine clinical practice can now be overcome with the advent of improved technology. Powerful software can now be accessed easily at the patient's bedside and analysed in real time via the internet,1 suggesting that this could be used in real practice to aid real-time decision-making, perhaps akin to the way an ECG is used in chest pain. Further work is planned in order to assess its inception and use in a modern day ED.
References
Supplementary materials
Online only appendix
Footnotes
Competing interests None.
Provenance and peer review Not commissioned; externally peer reviewed.