Article Text
PDF
Statistics from Altmetric.com
WHAT ARE THE NEW FINDINGS
We describe a novel, stepwise approach to rapidly screening opportunities to create emergency medicine research partnerships with private companies developing innovative medical devices, digital health tools and artificial intelligence algorithms.
The Stanford Emergency Medicine Partnership Program (STEPP) applies best practices from industry, including the use of a sales funnel and venture capital evaluation practuces, to centralize and streamline the evaluation and execution of innovative partnerships.
Over a 1.5-year period, STEPP evaluated 68 potential partners and initiated partnerships or collaborations with 10 of them; engaged 64 faculty, fellows, residents and students; and produced 9 peer-reviewed articles and conference presentations.
HOW MIGHT IT IMPACT HEALTHCARE IN THE FUTURE
STEPP has provided operational benefits, increased faculty and learner development, and improved engagement with industry to innovate in the emergency medicine space.
Our model could be adapted and applied to other healthcare departments seeking to promote innovation and improve operations to enhance patient care.
Introduction
Emergency departments (EDs) face a multitude of inefficiencies and quality issues including challenging triage processes, overcrowding and boarding. Health systems, in general, face challenges related to patient flow bottlenecks, long wait times and limited access to real-time data for decision-making.1 Hospital departments require innovative solutions to enhance patient care and optimise operations.2 This need grew more urgent with the COVID-19 pandemic, which illuminated large care gaps3 that threaten the safety of both patients and providers. Addressing these challenges is paramount for delivering timely and effective care to patients, not only in the ED, but in all hospital departments and healthcare systems.
Although various approaches have been pursued to address healthcare inefficiencies, they often involve individual stakeholders working in isolation, resulting in fragmented solutions that may not be widely generalisable, adopted or integrated into existing workflows. Stakeholders, including physicians, hospital administrators and technology partners, may build parallel, non-coordinated solutions with varying degrees of success and adoption. Meeting the current challenges in care delivery requires innovative solutions that can be rapidly integrated into healthcare workflows by departments and health systems.4
The past decade has seen a rapid expansion of novel healthcare technologies designed to increase access to care, reduce healthcare disparities, improve quality and reduce the cost of care.5 As the pace of healthcare technology accelerates, the ability to evaluate and implement solutions will need to evolve, a reality recognised by the US Food and Drug Administration that led them to rethink current drug approval pathways.6
Unfortunately, the time from development to broad implementation of new healthcare technologies is on the order of 10–20 years.7 As industry plays an increasing role in healthcare innovation and bringing new technologies to market, it is imperative that academic medical centres engage with industry to assess the utility and efficacy of these technologies, determine implementation pathways and provide critical feedback. Such partnerships may ultimately accelerate the uptake and effective implementation of beneficial healthcare technology.
Prior research on improving ED operations and implementing solutions has focused on upgrading triage and other ‘front-end’ processes,8–11 using simulation12–15 and improving performance metrics.16–19 Business fundamentals, like lean practices20–22 and increasing throughput,20 23–25 have also been also noted. However, several processes were challenging innovation engagement in our ED. First, we did not have a way for companies to interface with our department if they were interested in a partnership. Second, we did not have a standardised way of evaluating these companies when approached; in fact, a number of our faculty ended up pursuing partnerships that were ill-advised or not worth their time. Third, as the complexity of the healthcare system increased, we found that our faculty required help navigating the numerous hurdles involved in taking a partnership from concept to execution.
Our goal was to build a systematic, reproducible approach to evaluating and initiating academic–industry partnerships to expedite the implementation of beneficial healthcare technologies. We developed the Stanford Emergency Medicine Partnership Program (STEPP) as a collaborative resource to systematically evaluate, implement and optimise novel solutions—including advanced medical devices, digital health tools, artificial intelligence (AI) algorithms and telemedicine solutions—designed to address specific ED inefficiencies and quality concerns. Here, we describe the STEPP methodology and its application to identify opportunities to improve operations and patient care in the ED while also fostering a culture of innovation and collaboration.
Methods
Stakeholders
We sought to develop an approach to evaluating academic–industry partnerships that would align the interests and opinions of the various stakeholders, ensuring that their diverse perspectives are considered and ultimately leading to better-informed decisions and increased buy-in. With this rationale in mind, ED and hospital leadership comprising representatives from research, strategy, operations, digital health and education met monthly over the course of a year to design a new programme aimed at enhancing our ability to identify and execute innovative partnerships.
Physician involvement was based on a combination of self-selection and fulfilling leadership roles.
The STEPP model
As there are limited models for industry partnership evaluation in healthcare, we examined the best practices in business. Based on the parallels related to the evaluation of new ideas, we ultimately patterned STEPP after the process used by venture capital firms to evaluate companies for investment. Specifically, we incorporated and applied two best practices used in the venture capital industry—the pipeline funnel and company evaluation.
While there is no standardised approach to evaluate and determine what to invest in, most venture capital firms follow a general process that resembles a funnel (figure 1).26 This process involves several stages through which a large number of potential investments are evaluated and gradually narrowed down to a few select opportunities (typically <0.1%).27 After sourcing potential opportunities through active searching and accepting unsolicited pitches from entrepreneurs, the first step is the initial screening. Here, each opportunity is assessed based on various factors, including the problem space, product approach, go-to-market strategy and the team.
IPEC, Innovation Partnership Evaluation Committee; SME, subject matter expert.
For those opportunities that pass the initial screening, venture capital firms will conduct more in-depth research and analysis, for example, assessing the technology or intellectual property and verifying claims made. Venture capital firms may also seek input from industry experts or consult third-party sources to validate the potential for impact and return on investment. The next step is to decide whether to invest in the opportunity or not, which is often done collectively by the firm’s investment committee, which weighs the potential risks and rewards. If the venture capital firm decides to invest, they will negotiate the terms of the investment, and the deal is closed. They will also contribute to the company’s success as advisors, as board members, and with strategic introductions. Ultimately, they will report on the company’s progress to their investors.
We used this process as a guide to develop the STEPP model to help us evaluate a wide variety of disparate and often vague partnership opportunities for improving emergency medicine operational efficiency and patient care. This model was built to not only facilitate decision-making around these partnerships, but also to involve faculty, fellows, residents and students and provide experience in innovation and health technology, thus serving both operational and academic goals. This process and its relationship to the venture capital pipeline funnel are illustrated in figure 1 and described next.
Digital front door: potential partners reach out to our department through our digital front door, a website that describes STEPP and provides a contact form for companies to complete. We also access leads through pitch competitions, referrals and word of mouth. We use task management and project-tracking software to document the progress of companies in our funnel as they enter and then proceed through the various stages of evaluation and partnership.
Mining: our team uses this process to evaluate companies as potential partners. We prioritise their technology, the strength of their leadership team, the capitalisation of the company and their go-to-market plan. Although these characteristics are more commonly used in business, we realised that they can also indicate the likelihood that a partnership align with our mission to accelerate innovation in acute care to improve health. Mining consists of several core meetings designed to identify the most promising potential partners:
Screening meetings are used to learn about the company’s technology and determine if it is relevant to the ED and evaluate the potential for partnership. There are no costs for companies that participate in screening meetings and additional steps of the Mining process.
Subject matter expert (SME) meetings are used to further validate the company and technology. SMEs are members of our academic faculty and nursing leaders who are selected based on their unique areas of expertise. After these meetings, we not only obtain feedback from the SME on the company and its technology, but also ask if the SME would be interested in serving as a principal investigator (PI) for research with the potential partner.
Partnership specification meetings are used to scope out research protocols and pilot project opportunities. We conduct this meeting to detail the partnership goals, phases and costs. Our department invests faculty time, pays for expert internal staff consultations and funds department activities that can be justified with a return on investment based on scholarly work, research results and career development. Consequently, we also expect that the company partner will also invest time, money and resources into the collaboration.
Qualifying: for partnerships that will affect department operations, treatment protocols and integration with our electronic health record (EHR), additional approval is needed from the Innovation Partnership Evaluation Committee (IPEC). This group is composed of department leaders and faculty with industry and venture experience. IPEC approval is not required for collaborations that do not affect department operations or require EHR integration, such as feasibility and usability studies and product validation.
Team formation: when partnerships and collaborations are created, we create a STEPP team to execute our goals. While these teams vary in size based on the type of project, they consist of a STEPP Team Leader and an SME who serves as a PI. Additional team members may include representatives and research assistants from the ED research team.
Execution and broadcasting: this step involves the completion of the research collaboration or piloting of the new technology. We disseminate the results via academic publications, conference talks, poster sessions, our website and social media.
Tracking and reporting
The STEPP team meets regularly to review companies as they are moving through the funnel, determine if they should progress to the next stage and, if not, move them to either an ‘active tracking’ or ‘rejected’ status. We use ‘active tracking’ to continue communication with companies for possible future collaboration, while ‘rejected’ indicates companies no longer being considered. The STEPP team reports regularly on the programme’s progress to the ED executive committee. As the goal of STEPP is to streamline the evaluation and execution of academic–industry partnerships, we evaluated the number of partnerships as the primary outcome. Given STEPP’s relationship with an academic ED, scholarly output was also recorded. The programme’s reach within the ED was measured based on the number of faculty members and fellows who engaged with each portion of the funnel. The educational impact was evaluated based on the engagement of medical students, residents and fellows involved in the STEPP process.
Results
Between 1 July 2021 and 1 December 2022, the STEPP team screened 68 companies that sought partnership through the STEPP website or networking events (figure 2). Almost half of these companies (n=32) moved on to evaluation in an SME meeting. Of those, 12 were recommended for further evaluation in a partner specification meeting. Of these 12 potential partners, 8 were found to have a potential impact on clinical operations or involved EHR integration and were evaluated by IPEC. Collaborations were pursued with the other four companies, and these projects are in various stages of completion. Of the eight companies that gave IPEC presentations, six were cleared and are in various stages of implementation. The six partnerships and four collaborations that emerged from the STEPP funnel include efforts to pair AI and blood banking to identify new diagnostic biomarkers, develop monitoring devices that can identify intracranial haemorrhage without advanced imaging, a device that uses computer vision to focus room cleaning and improve patient turnover, and a bedside test that diagnoses and quantifies concussions, among others.
IPEC, Innovation Partnership Evaluation Committee; SME, subject matter expert.
Of the original 68 companies seeking partnerships, 58 did not move on to either SME evaluation (n=36) or partner specification meetings (n=20), and 2 were not cleared by IPEC (figure 2). Of these 58 companies, 43 continue to be actively tracked for future potential opportunities while 15 have been rejected from the STEPP process.
A total of 44 faculty members were engaged at various steps of the STEPP funnel to evaluate potential partners and their projects, participating as STEPP team members, SMEs or IPEC members (online supplemental table 1). In addition to faculty, 20 fellows, residents and medical students were members of the STEPP team, participated in SME calls and IPEC meetings, or engaged with STEPP during an elective rotation.
Supplemental material
The STEPP process resulted in three publications.28–30 In addition, eight abstracts were submitted for presentation, of which six were featured as talks and poster presentations at the annual meetings of the American Academy of Emergency Medicine, Association of American Medical Colleges Western Group, Society for Academic Emergency Medicine, Association for Medical Education in Europe and the Mediterranean Emergency Medicine Congress.
Discussion
STEPP has demonstrated considerable success in addressing contemporary healthcare challenges and fostering an environment of innovation within the ED. This success can be attributed to the effective adaptation of industry concepts such as sales funnels and venture capital evaluation methods, centralisation of partnership evaluation and implementation, and robust institutional support.
Our structured approach to evaluating healthcare innovations streamlined the implementation of technology partnerships, resulting in enhanced ED operations, increased faculty engagement and a wealth of opportunities for scholarship. Moreover, the six partnerships and four collaborations facilitated by STEPP have significantly contributed to modernising and advancing acute care through usability studies for medical devices, validation studies for AI algorithms and pilot studies for new operational tools.
Four specific lessons were learnt from the implementation of STEPP:
Leveraging industry concepts: the successful application of sales funnels and venture capital evaluation methods within an academic department context underscores their utility in providing a structured framework for evaluating and implementing innovative partnerships.
Centralising processes: by centralising the evaluation and execution of partnerships, we achieved operational benefits that streamline our department’s ability to tackle the complex, yet essential, aspects of remaining a forward-thinking and innovative institution.
Promoting faculty engagement: contrary to concerns that centralisation might limit faculty stakeholder participation, faculty participated in various aspects of STEPP, including serving as STEPP team members, SMEs and research partnership evaluation committee members, and authoring scholarly publications.
Early partnership implementation: STEPP projects had the most success when stakeholders from research and digital health were involved early. Involving these experts helped accelerate evaluation of each company and its technology, address project financing and ensure appropriate technology capabilities and approval. Early engagement of industry partners with key stakeholders can also help reduce friction and anticipate needs early in the process to improve implementation.
Our findings build on existing research that establishes the relevance of business concepts, such as lean principles and throughput analytics, for enhancing ED operations.20–22 Additionally, our contribution showcases the successful implementation of industry and innovation within the medical field.
Limitations
Certain factors unique to our department may have played a role in our success; for example, our location in Silicon Valley attracts startup capital and faculty with valuable industry connections. Our access to financial and faculty resources may not be universally replicable. However, many of the core concepts and strategies employed by STEPP can be adapted and applied to other hospital departments. For example, most departments already have physicians dedicated to quality improvement and they can use similar processes to evaluate and implement new technology. Healthcare companies across the country are pioneering products and would benefit from partnering with a local hospital to gain high-level physician feedback and pilot opportunities.
Not all health systems encourage innovation, and policies may need to be updated for it to flourish. For example, administration can establish technology transfer offices, support research and clinical trials, allow physicians to retain intellectual property and reward physicians professionally and financially for innovative contributions. Conversely, innovation is discouraged when research is not supported with funding and professional advancement, and when physicians are not able to retain intellectual property.31 32
Conclusion
STEPP is a novel, structured process for evaluating and implementing healthcare innovations in an academic ED, inspired by industry concepts such as the sales funnel and venture capital evaluation methods. STEPP streamlines the evaluation and execution of partnerships with companies offering medical devices, digital health tools, AI algorithms and operational solutions, addressing both quality improvement needs and department and health system goals. Centralising the evaluation and implementation of partnerships led to increased faculty engagement, enhanced educational opportunities and improved ED operations. Through its use of clear entry points, evaluation criteria, implementation processes and faster timelines, the programme has accelerated innovation in acute care through usability studies for medical devices, validation studies for AI algorithms and pilot studies for new operational tools. The programme strives to develop a win–win for health systems and industry companies through its use of clear entry points, evaluation criteria, implementation process and faster timelines. The success of STEPP demonstrates that adapting industry concepts and centralising innovation initiatives can be beneficial for healthcare departments seeking to improve their operations and resources.
Ethics statements
Patient consent for publication
Ethics approval
Not applicable.
Acknowledgments
The authors would like to thank Justin Larkin, MD, Partner, Bio+Health Investing Team at Andreessen Horowitz, for his insightful comments on the manuscript.
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
X @johnnyd
Contributors All authors were active participants in the development and execution of the STEPP model. JD drafted and edited the manuscript. MYAB reviewed and revised the manuscript and provided structural guidance. SS and MCS revised the manuscript. CR, GB and RR reviewed and revised the manuscript. All authors approved the final version of the manuscript for publication.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.