You are here

Article Text

Article menu

Download PDFPDF

XML
Health policy
Research
Scanning the horizon: a systematic literature review of methodologies
  1. Philip Hines1,2,3,
  2. Li Hiu Yu1,3,
  3. Richard H Guy1,4,
  4. Angela Brand2,3,
  5. Marisa Papaluca-Amati1
  1. 1 Scientific Committees Regulatory Science Strategy, European Medicines Agency, Amsterdam, The Netherlands
  2. 2 UNU-MERIT, United Nations University-Maastricht Economic and Social Research Institute on Innovation and Technology, Maastricht, The Netherlands
  3. 3 Department of International Health, Faculty ofHealth, Medicine and Life Sciences (FHLM), Maastricht University, Maastricht, The Netherlands
  4. 4 Department of Pharmacy and Pharmacology, University of Bath, Bath, UK
  1. Correspondence to Philip Hines; philip.a.hines{at}gmail.com

Abstract

Objectives Society is confronted with the rapid emergence of innovation in science and technology. To manage this, horizon scanning is being adopted globally to identify, assess and prioritise innovations and trends at an early stage of their development. This enables decision-makers to be better informed and to prepare for change. The aim of this paper is to systematically identify and evaluate horizon scanning methodologies employed in the healthcare and biomedical fields.

Methods A systematic literature review was performed using PubMed and Embase and was supplemented with grey literature searches (2008–2018). The principal methodologies used in horizon scanning were extracted.

Results Approximately 100 articles were summarised in a literature map. The search revealed many examples of horizon scanning across disciplines. Challenges, such as the need to refine prioritisation criteria, manage uncertainty inherent in the findings and improve the dissemination of identified issues, have been highlighted.

Conclusion Horizon scanning, when performed appropriately, is a flexible and potentially reliable tool, with a wide variety of methods. Horizon scanning can inform and influence decision-making, through identifying opportunities and challenges, from an organisational to an international level. Further research to identify the most effective methodologies available would add depth to this landscape and enable the evolution of best practice to most efficiently anticipate novel developments and innovations.

  • public health
  • qualitative research

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

https://doi.org/10.1136/bmjopen-2018-026764

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Strengths and limitations of this study

    • This systematic review offers an up-to-date perspective on horizon scanning methodologies—incorporating practices from a number of different fields; however, as reviews into horizon scanning in healthcare have been undertaken previously, some duplication of findings was inevitable.

    • A detailed evaluation, and a more practical guide to all the methodologies, could not be performed for practical reasons; specifically, the inconsistent reporting of the horizon-scanning details and their efficacy and the continually evolving approaches employed.

    • The paper may be subject to omission bias as the literature reported systematic methodological aspects of horizon scanning; therefore, some ad hoc horizon-scanning methods may not be captured.

    Introduction

    Across organisations worldwide, the rapid emergence of high impact innovation is a major challenge faced by decision-makers.1 To respond, the identification of future innovations and trends is being undertaken in a comprehensive, systematic and sustainable manner so that policymakers, and other stakeholders, can respond appropriately and enable innovations to reach the market with minimal developmental, legal, regulatory, process or procurement bottle-necks. To catalyse the achievement of this objective, horizon scanning is emerging as a valuable and viable strategy. This is particularly true in the health sector, where the European Medicines Agency (EMA) is seeking to promote the availability of innovative medicines using horizon scanning. This foresight will in turn inform the Agency’s Regulatory Science Strategy and the European medicines regulatory network strategy.2–4

    Horizon scanning has been in use for many years, initially by commercial organisations and later by public bodies; Japan was an early adopter of foresight methodologies in the 1970s.5 6 Since then, horizon scanning has been used across diverse sectors to aid financial, policy, process and research planning.7 8 There are many definitions of horizon scanning,9 10 but most can be captured by its generic characterisation as a systematic examination of information sources to detect early signs of important developments. The approach generally targets the early lifecycle of technologies—that is, in an early phase of adoption before their introduction onto the market—but may also scan for broader trends, challenges and opportunities. It provides an early warning of ‘signals’, rather than a comprehensive study of their impact.

    Horizon scanning generally follows a process of signal detection, filtration, prioritisation, assessment and dissemination (figure 1). Its use is growing across sectors,11 and this risks the duplication of efforts (both in design and execution) as similar stretches of the horizon are scanned by many.

    Figure 1
    Figure 1

    Common stages of horizon scanning from the EuroScan network.9 This figure is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License (CC BY-NC-SA 4.0).

    There have been no recent reviews of horizon scanning methods used in the health sector,9 12 or those looking at broader biomedical sectors.6 7 12–14 As a result, the aim of this literature review is to systematically identify and evaluate horizon scanning methodologies employed in the healthcare and biomedical fields. The overall goal is to broaden and update knowledge on the methodologies used, and through mapping and evaluation, provide a useful guide for the establishment and optimisation of future horizon scanning initiatives. This includes the activities of the EMA’s recently established Regulatory Science Observatory, as well other international efforts to reduce duplication including the EU Innovation offices Network, the European Network for Health Technology Assessment and the International Coalition of Medicines Regulatory Authorities (ICMRA).4 15

    Methods

    A systematic review of the literature was performed to capture and map the use of horizon scanning activity and the widespread use of modern IT/web capabilities, over the period from 2008 to 2018 (see online supplementary file 1).16 Three researchers were involved and a systematic protocol was followed to minimise inter-rater bias; this involved cross-validation between two researchers, with diverging opinions then arbitrated by the third. This systematic approach also permitted gaps and inconsistencies in the field to be identified. Data collection followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines17: the literature was first screened and filtered using inclusion and exclusion criteria; the accepted papers then underwent data extraction and analysis, and, finally, the scanning methodologies were mapped.

    Supplemental material

    Search strategy

    Medline and Embase bibliographic databases were searched to identify research papers on the use of horizon scanning and the methods used for this purpose. The final search took place on 4 July 2018. Grey literature and bibliographies of the most relevant research papers supplemented this search. The primary search terms used were derived from previous literature: ‘horizon scan*’ OR ‘strategic foresight*’ OR ‘systematic early dialogue*’ OR ‘early warning and alert system*’. All literature, of which the title or abstract contained any of the keywords above, was flagged.

    Inclusion and exclusion criteria

    Only articles published in English or Italian from 2008 to 2018 were included. In the first round of quality appraisal and screening (see online supplementary file 1), the publication abstracts or title had to indicate:

    1. A methodology for horizon scanning or strategic foresight, or a discussion or experience provided of horizon scanning;

    2. A breadth of horizon scanning of the relevant field or address a methodological aspect which may be generally applicable across different fields.

    In the second round, the full texts of the selected articles were then reviewed according to additional inclusion criteria:

    1. The horizon scan or foresight methodology was detailed.

    2. The priority areas included relevant science and/or technology.

    3. A collaborative/international approach was used

    4. The horizon scanning undertaken scanned a period of between 2 and 15 years ahead.

    Alternatively, the paper was required to demonstrate methodological aspect(s) of foresight or horizon scanning of potentially general applicability.

    The foresight period of 2–15 years reflected the fact that signals suggesting impact in less than 2 years concern innovations that are already in late-stage development, while those anticipated to ‘mature’ in 15–20 years’ time are too distant and uncertain to be useful. A collaborative/international approach was sought because of the global nature of innovation and change.

    The mapping was elaborated using the EuroScan International Information Network method, a scientific association of member organisations and individuals for the exchange of information on important emerging new drugs, devices, procedures, programmes and settings in healthcare (EuroScan), and novel methodological aspects from the searched literature.9

    Patient and public involvement

    No patients or public were directly involved in the design or conduct of the study.

    Results

    There were 413 papers identified through the initial database searches and grey literature, of which 252 were removed due to duplication or failing to meet the inclusion criteria. Sixty papers were excluded because of language issues or lack of access to full text. One hundred and one publications were included in this study after reading the full text as indicated (figure 2).

    Figure 2
    Figure 2

    Literature selection process flowchart.17

    Consistency between the selection of papers by the two researchers, who undertook the screening, was fair (Cohen’s unweighted Kappa=0.28).18 A third researcher therefore screened the excluded literature according to the criteria to ensure that all relevant papers were captured (and resulting in one further publication being selected).

    Most of the studies included in this review address horizon scanning as whole, following the process outlined in figure 1, and of these, many operate in a national context. A few papers also focus on the use of specific methodological aspects such as Delphi techniques. Given the databases interrogated, the most prevalent priority areas identified were environmental issues,9 gene therapy (8), oncology (9) and clinical practice. Public sector bodies in the UK, USA, Sweden and Australia published most often on the topic. Internationally, EuroScan was repeatedly referenced for its role in harmonising horizon scanning methodology, supporting its members and encouraging international collaboration.

    Box 1

    Information sources used for signal detection in horizon scanning

    Information sources (signal detection)

    • Scientific/biomedical literature review

    • Patents

    • Input from industry and industry associations

    • Other observatories

    • Media

    • International institutions and forums

    • Individuals, committees and expert groups

    • Surveys

    • Government bodies

    • Meetings and conferences

    • Other organisations

    • Grey literature

    Risk of bias was assessed in accordance with the BMJ guidelines and the Cochrane risk of bias tool.19 While bias was not typically found in these non-clinical qualitative studies, there were three exceptions:

    • A form of publication bias was likely in which only horizon scanning undertaken in organisations with a strong background in publishing academic publications and transparency were discovered. It was not possible to correct for this.

    • Omission bias may have occurred as the papers reported systematic methodological aspects of horizon scanning; however, some horizon scanning may occur in an ad hoc manner, for example, a signal discovered by word-of-mouth.

    • The competing interests of the authors were not considered beyond the standards of the source journals. This was because it was not thought highly relevant to the reporting of methodologies.

    Mapping

    The process outlined in the EuroScan Method toolkit was used as a basis and novel methodological aspects found in the literature were added.9 The resulting map is segmented into: signal identification and detection, criteria and methods of filtration and prioritisation, assessment, dissemination and updating of information and overall evaluation of the process. Notable references are given and the full data set is provided in the online supplementary file 2.

    Supplemental material

    Information sources and signal detection

    Signals are detected from manifold sources in a horizon scanning exercise; box 1 summarises the most common.

    Review of the scientific and biomedical literature is perhaps the most common place to identify innovation. Searches can be structured, using systematic and validated strategies, for broad or targeted topic areas20–22; a two-step approach, first to survey the field and, second, for a ‘deep dive’, can be used. Recently issued patents and published patent applications (eg, the European Patent Office)23 represent an alternative source of early signals, particularly of innovations originating in industry (large, medium and small). Systematic and/or ad hoc scanning of direct or indirect information about new findings from industry or industry associations24 25 (such as investment of venture capital in SMEs) is also useful for monitoring research pipelines. Similarly, other sources, such as clinical trials databases26 27 and intelligence gathered from research infrastructures and consortia, and from university and research institute technology transfer offices, are valuable.

    Additional observatories of potential value include repositories of innovation and trends generated from the horizon scanning outputs of international regulators and the committees and expert groups of governmental bodies, such as OECD and EuroScan.28–30 The media—print, electronic and social—generate substantial topic specific and commercially relevant information, available via RSS feeds,11 Twitter, Facebook and so on.31 Social media also provide signal suggestions from stakeholders. Workshops can also be useful to bring together diverse experts (chosen on the basis of their area of specialisation, breadth of knowledge, publications and commitment to the process) to discuss areas of novel science and technology and to collaboratively scan the horizon from different points of view.32 These ‘sand-pit’ exercises can be supplemented by participants from the scanning organisation itself, appropriate stakeholder groups, external consultants and policy-makers.33 34 For sustainable and continuous horizon scanning, it may prove valuable to create a steering committee, think tank or ‘idea radar’ including representatives of the aforementioned participants.12 35

    Delphi studies are widely used to pool knowledge and build consensus around emerging issues. There are two or more rounds involved. In the first, participants identify relevant issues, which are then pooled and ranked; the second round sees these issues discussed followed by their re-ranking. This process is iterated until a consensus is reached. Several Delphi variations have been described, from more conventional workshop formats to the use of online tools such as Nvivo (quantitative analysis of text) or Wordle.net (a word cloud tool).36 The design of a Delphi study should take into account the sample size and confounding factors, such as the level of conformity in the group.7 37 38

    Surveys, conducted via the web or by mail,12 25 enable staff of an organisation, stakeholders and the public to be asked to identify new technologies or trends.39 These may be most useful when horizon scanning in well-defined fields.7 Semistructured interviews covering a standard set of questions can also be used, with similar outcomes.21 However, public input was not found to be hugely productive in topic identification.40 Likewise, an attempt to establish a Wikipedia community has been largely unsuccessful.40

    Finally, a number of additional sources have been identified including draft legislation and policy papers from governmental bodies, the proceedings of scientific conferences and symposia, professional and scientific societies, interest groups, think tanks and research funders (government, charities, venture capital, etc), the so-called grey literature where global shifts that influence society, the economy and the environment—megatrends—41 are sometimes foreseen. Google alert queries, Google Trends, Google News Timeline, Google Insight and blogs were also mentioned.40 42

    Filtration criteria and methods

    Table 1 presents the criteria and methods commonly used to discard irrelevant signals. Several key criteria concerning a signal’s potential impact were used12 22 43–45 including what are the costs, and the cost–utility ratio, of resource consumption and what are the implications in terms of quality of life, burden of disease and patient safety45 46?

    View this table:
    Table 1

    Filtration criteria and methods used in horizon scanning to discard irrelevant signals

    The level of evidence is a further important criterion that has been ranked using a simple traffic light system,47 where green denotes sufficient evidence to support the uptake of the signal, yellow indicates insufficient evidence to support uptake but the evidence may constitute useful information and red implies unsupportive or insufficient evidence.

    In terms of filtration methods, these may be separated into those which tag signals according to the criteria in a binary yes/no fashion, or those which use distinct or graduated categories, for example, confirmed, likely, potential, unlikely and questionable.27 Automated text-mining tools can be used with databases to enable the identification, tagging and categorisation of signals and facilitate clustering and filtering.7 Individual or group filtration may be performed by organisational staff, who can also undertake some initial peer review that is ultimately performed in depth by external experts. The latter can also contribute to, or determine, the weighting of signals, according to the criteria, using an evidence framework.22 32

    Prioritisation criteria and methods for assessing signals

    The signals which have met the filtration criteria can then be prioritised. The prioritisation criteria which must be met, and the methods used to do so, are collected in table 2.

    View this table:
    Table 2

    Prioritisation criteria and methods used in horizon scanning to assess signals

    Logically, the criteria consider the potential impact on outcomes, a clear example being resource consumption, and the cost implications.12 43 The size and composition of the affected population are therefore important factors,12 46 48 49 as well as the expected variation that may be observed between different subsets.43 48 For the signal to be prioritised, the time frame must be realistic8 12 50 and there must be a clear, factual indication of true novelty and desirability. In addition to evidence of effectiveness, consideration must be given to the relative added value over current practice12 and whether this sufficient to satisfy strategic and/or political priorities and policies (eg, reduction in inequality).43

    With respect to prioritisation methods, a simple qualitative approach uses short summaries of the signals as a basis to prioritise.51 Quantitative or semiquantitative approaches are obviously more rigorous and typical.

    There were several novel Delphi approaches developed, for example, to acquire expert input online in a continuous feedback forum or market place. Here, participants prioritise, or purchase, a limited number of signals which then accrue a ‘price’ that can ultimately be used to prioritise those of greatest value.35 Controls are possible to counteract the possibility of scoring fatigue.52

    Finally, it should be emphasised that the engagement of experts for prioritisation must ensure diverse participation from different sectors, geographical regions, disciplines and demographics.31 44 53 Public consultation is a valuable asset to provide input and involvement from citizens and users in prioritisation and can be achieved in person, via email or online.12 40

    Signal assessment and methods

    The signals which have met the prioritisation criteria are then assessed. The factors assessed and the methods employed to do so are in table 3 (eg, in terms of resource implications and broad financial perspectives).

    View this table:
    Table 3

    Signal assessment and methods used in horizon scanning

    A key factor to consider in the assessment of any signal, of course, is the resource implications. The expected utilisation and availability of the innovation across different geographical regions is also important,12 34 46 as is an assessment of risk. A number of practical issues must also be considered, including actions needed to translate the signal into use (such as further research, the development of new processes and whether complementary technology, eg, is essential to realise the value of the signal), the time and investment required to do so, the need for new or specialised training of personnel involved,8 the cooperation and acceptance of key stakeholders, any ethical issues, access to the necessary experts7 and the intellectual property associated with the signal and whether legislative or regulatory guideline changes are required. As always, impact on the market must be taken into consideration. Is the innovation likely to have a disruptive effect, will it encounter reimbursement barriers, what are the timelines and milestones,35 etc? A consensus level of innovation can be sought (eg, important, moderate, modest).54

    Insofar as the methods used for signal assessment are concerned, a number of approaches are available.

    Dissemination and evaluation of the results of horizon scanning

    The key elements involved in disseminating and evaluating the results of horizon scanning are listed in table 4.

    View this table:
    Table 4

    Dissemination and evaluation of the results of horizon scanning

    In terms of dissemination, the assessment of an individual signal can be summarised in a document with the following elements: authors, lay summary, assessment objectives and methods, background and current practice, signal description, impacts and other issues, estimated time to impact, comparator signals (innovations), expert opinion and declaration of any conflict of interests. It may also be beneficial to include policy recommendations which are linked to decision-making priorities, structures and individual and cross-cutting policies.20 32 39 46 49 Dissemination can be achieved, when a new report is available, via numerous pathways, including email, social media, notification of target groups,40 44 public events involving the participation of policymakers,7 12 publicly accessible repositories of data or outputs that are clearly indexed, easily searchable and categorised, for example, by level of evidence and other metrics.42 55

    Dissemination of any new report should be made systematically through diverse platforms44 and shared directly with relevant organisations.42 The frequency of dissemination depends on circumstances.

    A related activity that bridges dissemination and evaluation is the updating of horizon-scanning information. This comprises four essential elements: (1) continually checking and pruning sources based on their usefulness, relevance and evolution,12 (2) monitoring and updating changes in signals by periodically refreshing the horizon scan,49 (3) reassessment of signals when sufficient new data are available or a step change in technology has occurred56 and (4) validating annually, for example, the horizon-scanning update by a team of expert researchers, practitioners and journalists.31

    Evaluation of the results of horizon scanning can be performed in the short, medium and long term.12 A short-term evaluation may involve the following actions: survey of an appropriate audience on the usefulness of horizon scanning in decision-making; use of metrics (eg, provided by Google Analytics), such as number of downloads, page views, average session duration, citations in publications and funding applications57; reports of failures; consistency with other horizon-scanning methods. In the medium term, an evaluation would include the responsiveness of the horizon-scanning team to requests, the ability to keep the horizon-scanning content up-to-date, comparing findings with other horizon-scanning agencies/databases (eg, EuroScan) and measuring sensitivity and associated predictive value. Finally, a long-term evaluation assesses the usage of horizon-scanning information in arriving at decisions, the accuracy of projections, the timeliness with which new technologies were detected and the prioritisation criteria which best signalled the impact of the technology.54

    A process and output audit represents another approach to evaluation and ensures the completeness of the search record, records of external input and expert contact details, clear filing of information used and a clear statement of the innovation in the briefing.6 A focus group of users can be employed to review the information input and dissemination and to develop a user-friendly interface through which to access a database.40 58

    Discussion

    There was a wide variety of sources and methods used to identify new and emerging issues. However, it was common to use scientific literature, individuals, committees and expert groups, the web and Delphi methodologies. That the scientific literature dominates is expected as innovation often begins in an academic environment and because widely accessible bibliographic databases have powerful search and filtering capabilities. The frequent use of the Delphi methodology may be explained by its ability to ‘crowd-source’ information and build a consensus among participants in a relatively short time frame. This consensus, particularly expert consensus, adds weight to the conclusions drawn from horizon scanning.

    Overall, the majority of the methods used were manual or semiautomated, with relatively few automated aspects. This could be due to the limited availability of software and budget constraints. Complex filtration, prioritisation and assessment criteria are some of the barriers to full automation that may be resolved in the not-too-distant future by the rapidly evolving fields of machine learning and artificial intelligence.59

    Dissemination of horizon-scanning reports appear to have rarely fed directly and systematically into policymaking. This may simply be a reflection of the unpredictable and political nature of policymaking, as well as a mismatch with the longer time scale of horizon scanning. Equally, it is probably fair to say that the information gathered by horizon scanning lacks, at least to some extent, the conventional measures of credibility and authority required to influence policymaking.32 New tools and approaches (eg, via generation of complex scenarios and the clear weighting of evidence)32 60 are probably needed to enable horizon scanning to be considered more seriously by policymakers.

    The distance of the horizon scanned was also found to be a tricky balance between the need to assess signals as early as possible to inform decision-makers and the limited information available at an early stage.56 There were many different evaluation methods employed, covering different time-spans, reflecting inter alia cultural differences, resource limitations and a time lag between horizon scanning and its critical evaluation (eg, a high false-positive rate of horizon scanning implied the need for tighter filtration criteria).14

    From a public policy standpoint, horizon scanning has both informative and creative functions, alerting policymakers to emerging issues and providing new, plausible policy options.7 This use of horizon scanning is well established for identification of emerging issues, both positive and negative, in global conservation and biological diversity.61 In some contrast, however, in the biomedical field, in which this review has concentrated, horizon scanning is biased towards identification of positive, innovative signals as those with low value inevitably have little impact.62 Finally, in addition to its institutional value, horizon scanning can significantly help related stakeholders, such as technology developers or civil society; it can reveal barriers to innovation and allow proactive engagement to reduce these barriers.43

    Limitations

    As systematic reviews into horizon scanning in healthcare have been undertaken previously, some duplication of findings was inevitable6 7 13 14 63; however, this review offers an up-to-date and wider perspective and includes methodologies from beyond the health field, for example, conservation. Resource limitations have precluded evaluation of horizon scanning in other, related sectors and consideration of material in languages other than English and Italian. Lastly, a detailed evaluation and a more practical guide to all the methodologies could not be performed for practical reasons: the inconsistent reporting of the horizon-scanning details and their efficacy and the continually evolving approaches employed. This effort must be viewed as a ‘snapshot’, therefore, of a rapidly moving target.

    Conclusions

    To respond to accelerating innovation, horizon-scanning methodology is being adopted both nationally and internationally, particularly in the public sector. The range of methods used and the limited assessment of their performance render recommendation of a single approach premature and explains why combining two or more techniques makes sense for validation and for improving the accuracy of predictions.6 7

    Undoubtedly, automation and the development of artificially ‘intelligent’ horizon scanning, which self-assesses and improves its signal management, are short-term milestones that will significantly improve the process, enhancing the evidence base, disseminating the acquired outputs efficiently, and facilitating decision-making. Self-evidently, given the need for horizon scanning across diverse disciplines, involving large numbers of interested stakeholders with related information needs, the process can only benefit from international collaboration. To this end, initiatives are underway, including within the ICMRA in which the EMA is taking an active role.15 64–66 Of course, scanning the horizon for signals is not an endpoint, in and of itself, but rather a window through which current and future opportunities and policies can be linked.67 It is essential, therefore, that further research be performed to develop, assess and ultimately implement the most efficacious methods of scanning and to ensure their acceptance and uptake by relevant stakeholders.

    Acknowledgments

    The authors are grateful to Monica Ensini and Lucia D’apote for their input and assistance.

    References

    1. 1.↵
      2. Collins A
      . The Global Risks Report 2018. Geneva: World Economic Forum, 2018.
    2. 2.↵
      2. Bujar MMN ,
      3. Liberti L
      . R&D Briefing 65: New drug approvals in six major authorities 2007 – 2016: Focus on the internationalisation of medicines. London: Centre for Innovation in Regulatory Science, 2017.
    3. 3.↵
      2. O’Dwyer L ,
      3. Nolan L ,
      4. Fisher C
      . Supporting Innovation through Regulation and Science: Ireland as an Innovation Hub for Health Products. Biomedicine Hub 2017;2:33.doi:10.1159/000481427
      OpenUrl
    4. 4.↵
      ICMRA. Key Outcomes. ICMRA Summit October. Kyoto: International Coalition of Medicines Regulatory Authorities (ICMRA), 2017.
    5. 5.↵
      2. Stevens A ,
      3. Packer C ,
      4. Robert G
      . Early warning of new health care technologies in the United Kingdom. Int J Technol Assess Health Care 1998;14:680–6.doi:10.1017/S0266462300011995
      OpenUrlCrossRefPubMed
    6. 6.↵
      2. Doos L ,
      3. Packer C ,
      4. Ward D , et al
      . Past speculations of the future: a review of the methods used for forecasting emerging health technologies. BMJ Open 2016;6:e010479.doi:10.1136/bmjopen-2015-010479
      OpenUrlAbstract/FREE Full Text
    7. 7.↵
      2. Amanatidou E ,
      3. Butter M ,
      4. Carabias V , et al
      . On concepts and methods in horizon scanning: Lessons from initiating policy dialogues on emerging issues. Science and Public Policy 2012;39:208–21.doi:10.1093/scipol/scs017
      OpenUrlCrossRefWeb of Science
    8. 8.↵
      2. Plüddemann A ,
      3. Heneghan C ,
      4. Thompson M , et al
      . Prioritisation criteria for the selection of new diagnostic technologies for evaluation. BMC Health Serv Res 2010;10:109.doi:10.1186/1472-6963-10-109
      OpenUrlCrossRefPubMed
    9. 9.↵
      EuroScan. A toolkit for the identification and assessment of new and emerging health technologies: Birmingham, 2014.
    10. 10.↵
      2. Thorne M
      . Sense-Making With Strategic Foresight: Scanning for Future Disruptions, 2018.
    11. 11.↵
      2. Urquhart GJ ,
      3. Saunders P
      . Wider horizons, wiser choices: horizon scanning for public health protection and improvement. J Public Health 2017;39:248–53.doi:10.1093/pubmed/fdw039
      OpenUrl
    12. 12.↵
      2. Sun F ,
      3. Schoelles K
      . A systematic review of methods for health care technology horizon scanning. Rockville (MD): AHRQ Publication, 2013:1–82.
    13. 13.↵
      2. Packer C ,
      3. Simpson S ,
      4. de Almeida RT
      . Euroscan international network member agencies: their structure, processes, and outputs. Int J Technol Assess Health Care 2015;31(1-2):78–85.doi:10.1017/S0266462315000100
      OpenUrl
    14. 14.↵
      2. Packer C ,
      3. Fung M ,
      4. Stevens A
      . Analyzing 10 years of early awareness and alert activity in the United kingdom. Int J Technol Assess Health Care 2012;28:308–14.doi:10.1017/S026646231200030X
      OpenUrlCrossRefPubMed
    15. 15.↵
      EUnetHTA. Horizon Scanning, Topic Identification, Selection and Prioritisation for European cooperation on HTA - Draft recommendations Brussels, 2018.
    16. 16.↵
      2. Mulrow CD
      . Systematic Reviews: Rationale for systematic reviews. BMJ 1994;309:597–9.
      OpenUrlFREE Full Text
    17. 17.↵
      2. Moher D ,
      3. Liberati A ,
      4. Tetzlaff J , et al
      . Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 2009;6:e1000097.doi:10.1371/journal.pmed.1000097
      OpenUrlCrossRefPubMed
    18. 18.↵
      2. McHugh ML
      . Interrater reliability: the kappa statistic. Biochem Med 2012;22:276–82.
      OpenUrl
    19. 19.↵
      2. Higgins JP ,
      3. Altman DG ,
      4. Gøtzsche PC , et al
      . The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ 2011;343:d5928.doi:10.1136/bmj.d5928
      OpenUrlFREE Full Text
    20. 20.↵
      CADTH. CADTH Environmental Scan Process. Ottawa: CADTH, 2015.
    21. 21.↵
      2. Jones MM ,
      3. Hall A ,
      4. Brooker D , et al
      . The Future of public health: A horizon scan. Rand Health Q 2014;4:18.
      OpenUrl
    22. 22.↵
      2. Varela-Lema L ,
      3. De La Fuente-Cid R ,
      4. López-García M
      . Developing a prioritized list of innovative technologies: the Spanish experience. Int J Technol Assess Health Care 2014;30:626–33.doi:10.1017/S0266462314000774
      OpenUrl
    23. 23.↵
      EPO. The European Patent Office. 2018 https://www.epo.org/index.html (accessed 06 Aug 2018).
    24. 24.↵
      2. Pharmascan U
      . Working together to improve NHS planning for new medicines: UK PhramaScan. 2018 https://www.ukpharmascan.org.uk/static/about
    25. 25.↵
      2. Smith J ,
      3. Ward D ,
      4. Michaelides M , et al
      . New and emerging technologies for the treatment of inherited retinal diseases: a horizon scanning review. Eye 2015;29:1131–40.doi:10.1038/eye.2015.115
      OpenUrlCrossRefPubMed
    26. 26.↵
      2. Duchesne GM ,
      3. Grand M ,
      4. Kron T , et al
      . Trans Tasman Radiation Oncology Group: Development of the Assessment of New Radiation Oncology Technology and Treatments (ANROTAT) Framework. J Med Imaging Radiat Oncol 2015;59:363–70.doi:10.1111/1754-9485.12255
      OpenUrl
    27. 27.↵
      2. Noorlander CW ,
      3. Kooi MW ,
      4. Oomen AG , et al
      . Horizon scan of nanomedicinal products. Nanomedicine 2015;10:1599–608.doi:10.2217/nnm.15.21
      OpenUrl
    28. 28.↵
      2. Clyne M ,
      3. Schully SD ,
      4. Dotson WD , et al
      . Horizon scanning for translational genomic research beyond bench to bedside. Genet Med 2014;16:535–8.doi:10.1038/gim.2013.184
      OpenUrlCrossRefPubMed
    29. 29.↵
      2. Saunders PJ ,
      3. Middleton JD ,
      4. Rudge G
      . Environmental Public Health Tracking: a cost-effective system for characterizing the sources, distribution and public health impacts of environmental hazards. J Public Health 2017;39:506–13.doi:10.1093/pubmed/fdw130
      OpenUrl
    30. 30.↵
      2. Varela-Lema L ,
      3. Punal-Riobóo J ,
      4. Acción BC , et al
      . Making processes reliable: a validated pubmed search strategy for identifying new or emerging technologies. Int J Technol Assess Health Care 2012;28:452–9.doi:10.1017/S0266462312000578
      OpenUrlPubMed
    31. 31.↵
      2. Sutherland WJ ,
      3. Broad S ,
      4. Caine J , et al
      . A Horizon Scan of Global Conservation Issues for 2016. Trends Ecol Evol 2016;31:44–53.doi:10.1016/j.tree.2015.11.007
      OpenUrl
    32. 32.↵
      2. Garnett K ,
      3. Lickorish FA ,
      4. Rocks SA , et al
      . Integrating horizon scanning and strategic risk prioritisation using a weight of evidence framework to inform policy decisions. Sci Total Environ 2016;560-561:82–91.doi:10.1016/j.scitotenv.2016.04.040
      OpenUrl
    33. 33.↵
      2. Sutherland WJ ,
      3. Freckleton RP
      . Making predictive ecology more relevant to policy makers and practitioners. Philos Trans R Soc Lond B Biol Sci 2012;367:322–30.doi:10.1098/rstb.2011.0181
      OpenUrlCrossRefPubMed
    34. 34.↵
      2. Veenstra DL ,
      3. Piper M ,
      4. Haddow JE , et al
      . Improving the efficiency and relevance of evidence-based recommendations in the era of whole-genome sequencing: an EGAPP methods update. Genet Med 2013;15:14–24.doi:10.1038/gim.2012.106
      OpenUrlCrossRefPubMed
    35. 35.↵
      2. Masum H ,
      3. Ranck J ,
      4. Singer PA
      . Five promising methods for health foresight. Foresight 2010;12:54–66.
      OpenUrlCrossRef
    36. 36.↵
      2. Dawson MN ,
      3. Algar AC ,
      4. Antonelli A , et al
      . An horizon scan of biogeography. Front Biogeogr 2013;5.
    37. 37.↵
      2. Birko S ,
      3. Dove ES ,
      4. Özdemir V
      . Evaluation of nine consensus indices in delphi foresight research and their dependency on delphi survey characteristics: a simulation study and debate on delphi design and interpretation. PLoS One 2015;10:e0135162.doi:10.1371/journal.pone.0135162
    38. 38.↵
      2. Birko S ,
      3. Dove ES ,
      4. Özdemir V
      . A Delphi technology foresight study: mapping social construction of scientific evidence on metagenomics tests for water safety. PLoS One 2015;10:e0129706.doi:10.1371/journal.pone.0129706
    39. 39.↵
      2. Kark S ,
      3. Sutherland WJ ,
      4. Shanas U , et al
      . Priority questions and horizon scanning for conservation: a comparative study. PLoS One 2016;11:e0145978.doi:10.1371/journal.pone.0145978
    40. 40.↵
      2. Simpson S ,
      3. Cook A ,
      4. Miles K
      . Patient and public involvement in early awareness and alert activities: an example from the United Kingdom. Int J Technol Assess Health Care 2018;34:10–17.doi:10.1017/S0266462317004421
      OpenUrl
    41. 41.↵
      2. Reimers-Hild C
      . Strategic foresight, leadership, and the future of rural healthcare staffing in the United States. JAAPA 2018;31:44–9.doi:10.1097/01.JAA.0000532119.06003.12
      OpenUrl
    42. 42.↵
      2. Gwinn M ,
      3. Grossniklaus DA ,
      4. Yu W , et al
      . Horizon scanning for new genomic tests. Genet Med 2011;13:161–5.doi:10.1097/GIM.0b013e3182011661
      OpenUrlCrossRefPubMed
    43. 43.↵
      2. Ciani O ,
      3. Jommi C
      . The role of health technology assessment bodies in shaping drug development. Drug Des Devel Ther 2014;8:2273–81.doi:10.2147/DDDT.S49935
      OpenUrlPubMed
    44. 44.↵
      2. Nachtnebel A ,
      3. Breuer J ,
      4. Willenbacher W , et al
      . Looking back on 5 years of horizon scanning in oncology. Int J Technol Assess Health Care 2016;32:54–60.doi:10.1017/S0266462316000052
      OpenUrl
    45. 45.↵
      2. Stafinski T ,
      3. Topfer LA ,
      4. Zakariasen K , et al
      . The role of surgeons in identifying emerging technologies for health technology assessment. Can J Surg 2010;53:86–92.
      OpenUrlPubMed
    46. 46.↵
      2. Eriksson I ,
      3. Wettermark B ,
      4. Persson M , et al
      . The early awareness and alert system in Sweden: history and current status. Front Pharmacol 2017;8:674.doi:10.3389/fphar.2017.00674
      OpenUrl
    47. 47.↵
      RANZCR. Position Paper - Techniques and Technologies in Radiation Oncology 2015. Sydney: The Royal Australian and New Zealand College of Radiologists, 2015.
    48. 48.↵
      2. Nachtnebel A ,
      3. Geiger-Gritsch S ,
      4. Hintringer K , et al
      . Scanning the horizon: development and implementation of an early awareness system for anticancer drugs in Austria. Health Policy 2012;104:1–11.doi:10.1016/j.healthpol.2011.11.003
      OpenUrlPubMed
    49. 49.↵
      2. Raman G ,
      3. Wallace B ,
      4. Patel K , et al
      . Update on horizon scans of genetic tests currently available for clinical use in cancers. Rockville (MD): Quality AfHRa, 2011.
    50. 50.↵
      2. Wild C ,
      3. Simpson S ,
      4. Douw K , et al
      . Information service on new and emerging health technologies: identification and prioritization processes for a European union-wide newsletter. Int J Technol Assess Health Care 2009;25:48–55.doi:10.1017/S0266462309990687
      OpenUrlCrossRefPubMed
    51. 51.↵
      2. Maddern G ,
      3. Boult M ,
      4. Ahern E , et al
      . ASERNIP-S: international trend setting. ANZ J Surg 2008;78:853–8.doi:10.1111/j.1445-2197.2008.04679.x
      OpenUrlPubMed
    52. 52.↵
      2. Chapman AM ,
      3. Taylor CA ,
      4. Girling AJ
      . PRM22 The headroom method of early economic evaluation of medical devices: a useful tool for device developers? Value in Health 2012;15:A463–4.doi:10.1016/j.jval.2012.08.1486
      OpenUrl
    53. 53.↵
      2. Rudd MA ,
      3. Moore AFP ,
      4. Rochberg D , et al
      . Climate research priorities for policy-makers, practitioners, and scientists in Georgia, USA. Environ Manage 2018;62:190–209.doi:10.1007/s00267-018-1051-4
      OpenUrl
    54. 54.↵
      2. Joppi R ,
      3. Demattè L ,
      4. Menti AM , et al
      . The Italian Horizon Scanning Project. Eur J Clin Pharmacol 2009;65:775–81.doi:10.1007/s00228-009-0666-z
      OpenUrlPubMed
    55. 55.↵
      2. Khoury MJ ,
      3. Gwinn M ,
      4. Dotson WD , et al
      . Knowledge integration at the center of genomic medicine. Genet Med 2012;14:643–7.doi:10.1038/gim.2012.43
      OpenUrlCrossRefPubMed
    56. 56.↵
      2. Wild C ,
      3. Langer T
      . Emerging health technologies: informing and supporting health policy early. Health Policy 2008;87:160–71.doi:10.1016/j.healthpol.2008.01.002
      OpenUrlCrossRefPubMed
    57. 57.↵
      2. Gomes PT ,
      3. Teixeira Vidal A ,
      4. Souza A
      . Radar – An Important Tool for Horizon Scanning Dissemination in Brazil. Value in Health 2017;20:A906.doi:10.1016/j.jval.2017.08.2787
      OpenUrl
    58. 58.↵
      2. Sun F ,
      3. Bruening W ,
      4. Uhl S , et al
      . Quality, regulation and clinical utility of laboratory-developed molecular tests. Rockville, MD, 2010.
    59. 59.↵
      2. Observatory I
      . Who We Are And What We Do. Newcastle upon Tyne, UK: National Institute for Health Research, 2018. http://www.io.nihr.ac.uk/what-we-do/
    60. 60.↵
      2. Gale P ,
      3. Breed AC
      . Horizon scanning for emergence of new viruses: from constructing complex scenarios to online games. Transbound Emerg Dis 2013;60:472–4.doi:10.1111/j.1865-1682.2012.01356.x
      OpenUrl
    61. 61.↵
      2. Sutherland WJ ,
      3. Butchart SHM ,
      4. Connor B , et al
      . A 2018 Horizon scan of emerging issues for global conservation and biological diversity. Trends Ecol Evol 2018;33:47–58.doi:10.1016/j.tree.2017.11.006
      OpenUrl
    62. 62.↵
      2. Mundy L ,
      3. Hiller J ,
      4. Merlin T
      . The true role of horizon scanning in Australia: Who it informs and why. Int J Technol Assess Health Care 2011;27:95–6.doi:10.1017/S0266462310001212
      OpenUrlPubMed
    63. 63.↵
      2. Sun F ,
      3. Bruening W ,
      4. Uhl S , et al
      . Quality, regulation and clinical utility of laboratory-developed molecular tests. Rockville, 2010.
    64. 64.↵
    65. 65.↵
      HMA, EMA. Mandate of the European Innovation Network, 2016.
    66. 66.↵
      2. Lepage-Nefkens IDK ,
      3. Mantjes G ,
      4. de Graaf G , et al
      . Horizon scanning for pharmaceuticals: Proposal for the BeNeLuxA collaboration. Brussels: Health Services Research (HSR) Brussels: Belgian Health Care Knowledge Centre (KCE), 2017.
    67. 67.↵
      2. Wettermark B ,
      3. Persson ME ,
      4. Wilking N , et al
      . Forecasting drug utilization and expenditure in a metropolitan health region. BMC Health Serv Res 2010;10:128.doi:10.1186/1472-6963-10-128
      OpenUrlPubMed

    Footnotes

    • Contributors All authors contributed to the conception or design of the work; the acquisition, analysis, or interpretation of data; drafting and revising the paper for intellectual content; final approval of the version to be published and agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

    • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

    • Disclaimer The views expressed in this article are the personal views of the authors and may not be understood or quoted as being made on behalf of or reflecting the position of the agencies or organisations with which the authors are affiliated.

    • Competing interests None declared.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data sharing statement The data set (excel) has been made available through being uploaded as supplementary information.

    • Patient consent for publication Not required.