Elsevier

The Lancet

Volume 374, Issue 9695, 26 September–2 October 2009, Pages 1105-1112
The Lancet

Series
No surgical innovation without evaluation: the IDEAL recommendations

https://doi.org/10.1016/S0140-6736(09)61116-8Get rights and content

Summary

Surgery and other invasive therapies are complex interventions, the assessment of which is challenged by factors that depend on operator, team, and setting, such as learning curves, quality variations, and perception of equipoise. We propose recommendations for the assessment of surgery based on a five-stage description of the surgical development process. We also encourage the widespread use of prospective databases and registries. Reports of new techniques should be registered as a professional duty, anonymously if necessary when outcomes are adverse. Case series studies should be replaced by prospective development studies for early technical modifications and by prospective research databases for later pre-trial evaluation. Protocols for these studies should be registered publicly. Statistical process control techniques can be useful in both early and late assessment. Randomised trials should be used whenever possible to investigate efficacy, but adequate pre-trial data are essential to allow power calculations, clarify the definition and indications of the intervention, and develop quality measures. Difficulties in doing randomised clinical trials should be addressed by measures to evaluate learning curves and alleviate equipoise problems. Alternative prospective designs, such as interrupted time series studies, should be used when randomised trials are not feasible. Established procedures should be monitored with prospective databases to analyse outcome variations and to identify late and rare events. Achievement of improved design, conduct, and reporting of surgical research will need concerted action by editors, funders of health care and research, regulatory bodies, and professional societies.

Introduction

Development and evaluation of surgical and interventional techniques proceeds through stages similar to those for drug development, but with important differences.1 In general, however, the appropriate model for surgery has probably more in common with complex interventions in areas such as psychological and physical therapies.

The UK Medical Research Council (MRC) set out guidance for the assessment of complex interventions in 2000,2 which was updated in 2008.3 Complex interventions are defined as methods consisting of several interacting components or involving the use of difficult or complex techniques, which may be applied in various ways. These are also defining characteristics of surgical procedures, but surgery has a specific combination of attributes that causes additional problems. We outlined these issues in the first two papers of the Series,1, 4 and they have guided the development of these recommendations.

The MRC recommendations include: development and evaluation through iterative phases; use of experimental rather than observational designs whenever possible; measurement of outcomes as well as process; reporting detailed descriptions of interventions to improve reproducibility, evidence synthesis, and wider implementation. We have tried to tailor these recommendations to the surgical setting.

We know that, if our proposals are going to improve the quantity and quality of surgical research, they should be practical and not create issues for the continuing development of procedures. Unrealistically demanding standards could hinder surgical innovation. In the short term, we cannot change how surgical innovation happens and so we need to adapt our methods to the process rather than doing the opposite.

Recommendations are futile without a viable mechanism to enable their adoption. We therefore address the roles that funding bodies, regulators, and journal editors could have in encouraging improvements in the conduct and reporting of surgical evaluation. We recognise that these roles, together with the attitudes of the surgical community and the public, vary worldwide depending on culture and legislation, resulting in differences in the process.

Surgical innovation and the factors affecting its evaluation have been described in the first two papers of this Series.1, 4 Here, we address the difficulties in the assessment of surgical innovations and propose some possible solutions. We focus on the evaluation of new invasive techniques and procedures rather than changes in associated aspects of care, such as diagnostic imaging or accelerated recovery programmes.

Section snippets

Stages in the development and assessment of surgical innovations

In the first paper of this Series,1 we describe how surgical innovation happens and how innovations are adopted. By contrast with the formalised approach for drug development, the process in surgery has been unregulated, unstructured, and variable. Nevertheless, it seems to proceed in phases;5 we have developed this idea in a descriptive model delineating stages of innovation, development, exploration, assessment, and long-term study (the IDEAL model, see table).

Stage 1 of innovation happens

Stage 1: innovation

The stage of innovation describes the first use of a new procedure in a patient, prompted by the need for a new solution to a clinical problem. This situation might occur in an emergency (eg, the development of damage control surgery for polytrauma11) or in a patient whose condition allows time for planning. If time allows, we suggest that the surgeon informs the hospital of the intention to undertake a new procedure. At this stage, research ethics approval is not appropriate, although full and

Stage 2a: development

Development involves the planned use of a procedure in an initial small group of patients (rarely more than 30 and sometimes less than ten) to support experience with its first use and often to refine or modify the precise technique. A good example is the report of tissue engineered autologous grafts for haemodialysis.7 The traditional method of reporting this experience in retrospective case series studies has been justifiably criticised.13 Instead, we recommend that protocols for prospective

Stage 2b: exploration

Exploration occurs once the procedure has been described and the main technical aspects worked out. Experience with the procedure may still be scarce, however, and outcomes with larger numbers of patients are usually needed (up to a few hundred) before a randomised clinical trial that compares the new procedure with traditional management is feasible. At this stage, the procedure is likely to be adopted by surgeons in more than one unit, making the issues of mentoring and learning-curve

Stage 3: assessment

Previous stages focused on the development of a new technique and the description of its outcomes; this stage aims to assess effectiveness against current standards. The new method should now be sufficiently evolved to warrant full evaluation, which does not mean that it will not evolve further. This stage should be seen as a milestone on a learning path; the key issue is to decide which is the best feasible comparator for the new procedure.

Randomised trials should be the default option in this

Stage 4: long-term study

In this stage, established procedures are assessed for rare and long-term outcomes, and for variations in outcome. Learning curves may be less important now than in previous stages, but differences in selection criteria or in the quality of surgery or aftercare may become apparent through unexpected outcome variation between study centres. The typical study design is a registry: making registries disease-based in this setting may be impractical, in which case careful monitoring of indications

Discussion

The IDEAL model is based on the suggestion that surgical innovation and evaluation can and should evolve together in an ordered manner from concept, through exploration, to validation by randomised trials (panel 1). This order does not always reflect real situations, in which the timing and nature of evaluation might depend on the type of development. Gradual evolution of a technique may culminate in a systematically different approach, as in delayed surgical intervention for infected

References (37)

  • P Craig et al.

    Developing and evaluating complex interventions: the new Medical Research Council guidance

    BMJ

    (2008)
  • P McCulloch

    Developing appropriate methodology for the study of surgical techniques

    J R Soc Med

    (2009)
  • ED Flora et al.

    A review of natural orifice transluminal endoscopic surgery (NOTES) for intra-abdominal surgery: experimental models, techniques, and applicability to the clinical setting

    Ann Surg

    (2008)
  • M Degiuli et al.

    Survival results of a multicentre phase II study to evaluate D2 gastrectomy for gastric cancer

    Br J Cancer

    (2004)
  • L Sjöström et al.

    Effects of bariatric surgery on mortality in Swedish obese subjects

    N Engl J Med

    (2007)
  • BE Keogh et al.

    Fifth national adult cardiac surgical database report 2003

    (2004)
  • JM Burch et al.

    Abbreviated laparotomy and planned re-operation for critically injured patients

    Ann Surg

    (1992)
  • The UK Confidential Reporting Programme for Aviation and Maritime

  • Cited by (1357)

    • Biofluorescence in surgery: Present and future

      2024, Seminars in Colon and Rectal Surgery
    View all citing articles on Scopus
    View full text