Article Text
PDF
Abstract
Background Hackathons aim to develop solutions to preidentified problem domains and catalyse startup cultures. Recently, the teaching and learning potential of hackathons has also been documented. In this study, we make the case for utilisation of hackathons as an alternative teaching and learning tool geared towards entrepreneurship and as an opportunity for interprofessional integration.
Methods This research study followed up with participants from the third hackathon at the Aga Khan University in Karachi, Pakistan. Hack MedEd was about solutions to problems of undergraduate and postgraduate medical education with an emphasis on low-income to middle-income countries. Participant evaluation data were filled at the end of the hackathon and gathered from three focused group discussions (FGDs): immediately before and after the event, a delayed follow-up after 11 months was recorded.
Results Of 116 participants, the majority (71%) were under 30 years old, and over half were female. The evaluations provided by hackers were positive overall with a mean score of 4.37 out of 5 on a Likert Scale. During the FGDs, participants spoke positively of the process and felt that, by the end of the hackathon, they had learnt something new. In the delayed follow-up FGD, teams that had undergone incubation expressed that they had gained a critical and simple skillset that they might not have acquired otherwise.
Conclusion Hackathons business incubation programmes may be considered an alternative teaching and learning tool—especially for individuals studying or working within the healthcare discipline within low-resource settings.
- assistive technology
- inventions
- economics
Statistics from Altmetric.com
Introduction
The term hackathon is a juxtaposition of the root words ‘hacking’ and ‘marathon’—an event initially brought into existence by software and hardware developers.1 After its introduction by the Massachusetts Institute of Technology in 2011, the healthcare hackathon has gained increasing popularity for its functionality in identifying potential problems and then creating concrete solutions via multidisciplinary collaboration.2 3
With a 3-year experience of conducting hackathons at the Aga Khan University (AKU) in Karachi, Pakistan, we have identified its potential in significantly improving interdisciplinary collaboration and formulating relevant solutions to problems faced by both patients and healthcare professionals.4–6 Through its dynamic problem-solving approach in proposing implementable ideas, the hackathon model has shown great capacity to accelerate outcomes.
The utility of a hackathon lies in converting a linear process of production into a circle that continually undergoes a cycle of change under perspectives from different fields.7 In the context of healthcare, it puts the healthcare professional at the forefront of innovation, and together with other professionals, enables him/her to arrive at a solution that works most appropriately in a setting he/she understands best.8 This ‘cross-pollination’ approach—leading to ‘disruptive innovation’—is fast-becoming a hallmark of the hackathon.1
The utility of the hackathon model as a tool for integration of medical education and other fields was first proposed by Aungst in 2015.9 In this paper, we seek to gauge the hackathon’s potential as a model for integration of different professionals working at our University Hospital, and its utility as an educational tool for technical and entrepreneurial skills, in the context of our participants’ experience of the first undergraduate and postgraduate level medical education hackathon of its kind in Pakistan.
Materials and methods
Study design and setting
We conducted a mixed methods (quantitative and qualitative) study on the third hackathon organised at AKU in February 2018, a major quaternary-care teaching hospital in Pakistan. The event’s name was Hack MedEd, with the theme to hack medical education, and the purpose to target both undergraduate and postgraduate level medical programmes at AKU. The event was attended by participants from AKU and other institutes.
Main outcome
Our main outcome was assessing participants’ responses. We achieved this through analysing immediate postevent evaluation survey responses and tabulating them as quantitative data. We triangulated these data through three qualitative focused group discussions (FGD)—immediately before, after and 11 months after the event. These discussions attempted to assess the impact of the hackathon on individual experience and growth of participants.
Inclusion and exclusion criteria
All participants (‘hackers’) at Hack MedEd who completed the postevent survey were included in the study. For the FGDs, only those who volunteered after an email was sent out to all participants, were then called in. Participants who were also simultaneously taking part in the event organisation were excluded from the study.
Study procedure/protocol
Hackathon 3.0, Hack MedEd, was a 2-day event that was organised by the Critical Creative Innovative Thinking forum that originated at AKU with the mission to channel innovation through creativity, critical thinking and collaboration, along with faculty leads of the undergraduate medical programme. After a call for participation, individuals who signed up for Hack MedEd attended the hackathon. The first day consisted of participants forming groups and working on a set of problems, with the help and guidance of their mentors. On the second day, the groups finalised the solutions to their chosen set of problems. This intense 2-day mental exercise was punctuated by scheduled bursts of fun and food, to keep energy levels upbeat.
The FGDs were conducted with the participants—one before and one after the event, to assess their expectations and impression after the 2-day rigorous activity.
After the event, the participants were also required to fill in evaluation forms. Table 1 outlines the questions asked. Participants were expected to score each statement on a Likert scale (1=most unfavourable; 5=most favourable). Data pertaining to participants’ demographics, affiliations and experience of the hackathon, were recorded and served as the quantitative end-process description of the hackathon and how it impacted the participants, that is, postevent individual appraisal.
Demographic characteristics of the 116 participants (hackers) at the third hackathon, Hack MedEd, at the Aga Khan University, Karachi, Pakistan, 17, 18 and 23 February 2018
A third FGD was held at an interval of 11 months to gauge progress of teams whose projects underwent business incubation. All FGDs were dual moderated mini FGDs, that is, they consisted of 2–5 participants and the discussion was aided by two facilitators (SS and KG). Each lasted an hour and was recorded and, later, transcribed. The quantitative data gathered from hackathon evaluations serve as the primary outcomes of our study. The findings of both FGDs are presented as qualitative data, and these served as the secondary outcomes of our study. The questions asked of the participants in the three FGDs can be found in the online supplemental file.
Supplemental material
Data analysis
Data were entered and analysed by using SPSS statistical package V.21. Participant demographics and professional affiliations were analysed using frequencies for categorical variables. The posthackathon survey scores for each question are presented as means and SD. The FGD transcripts were read independently by two authors and relevant content included in the manuscript. This allowed for data triangulation through the mixed methods (qualitative-quantitative) approach.
Results
Participant demographics and post event satisfaction
In total, 116 persons participated in Hack MedEd, of whom the majority (55 %) were females (table 1). Over two thirds of the participants (71 %) were less than 30 years old and over three quarters of the hackers were from within AKU (table 1). An overwhelming majority (102 of 116; 88 %) of the hackers were medical students, physicians or allied healthcare staff (table 2).
Occupational or professional distribution of the 116 participants (hackers) at the third hackathon, Hack MedEd, at the Aga Khan University, Karachi, Pakistan, 17, 18 and 23 February 2018
In terms of evaluating the event, 78% of the participants (90 of 116) responded with their thoughts and opinions (table 3). The participants rated the event an average score of 4.37/5. The participants were critical of the fact that a majority of participants were healthcare providers and rated the relevant category 3.78/5. An overwhelming majority of the hackers indicated that they would recommend our next hackathon to their peers or colleagues (87 of 90; 96.7%).
Evaluation questionnaire administered to the hackers at the third hackathon, Hack MedEd, at the Aga Khan University, Karachi, Pakistan, 17, 18 and 23 February 2018
The event also saw 11 mentors voluntarily take time to help teams dissect their problems and develop their solutions. All mentors had a reputable track record of being directly involved in medical education. We had a few faculty members from different departments, resident trainees, directors, Information and Technology (IT) professionals and company founders. At the end of the event, mentors were also asked to evaluate the event.
Focused group discussion: thoughts before the hackathon
As part of the pre-event FGD, participants were asked to verbalise their motivation for participating in the event. One described it as ‘a fusion between…interacting with people from different backgrounds and then working towards a single idea that would be pursued by all of them together’. In the words of another, ‘I wanted to participate to break barriers and think of new solutions, to come up with new ideas so that we can… have a structure for educating not only medical students but also the other people who have graduated and started different programs’. The participants were then questioned about their expectations of the hackathon,
‘What I’m expecting out of this hack is the ability to analyze my own ideas, and maybe find a common ground between mine and someone else’s who I feel would be able to really impact medical education as a whole’.
A general feeling among the group was expressed by one participant in saying that ‘…less money is spent on the way the workforce is catered for in healthcare and on the education by which a workforce is created, the doctors, the nurses’ which may be why medical education is not a priority, and the goal of this hackathon is to ‘bring that discussion of a change in medical education to the right people’.
Focused group discussion: thoughts after the hackathon
During the immediate postevent FGD, a similar strategy was used. We asked the participants about the pitching of, and evolution of ideas on their first day. One reported that he ended up with a different pitch than what he started with, and the concept was ‘about tackling the same problem but in a different and simpler way’. Most felt that the experience was gratifying because ‘there are number of platforms for discussing problems, but when it comes to solutions then we do not have any platform to suggest or to propose a solid solution’ and the hackathon provided this platform.
We discussed the impact of diversity among team members, and all participants echoed the need for, and importance, of diversity: ‘this was a platform where we met with a number of people because this is not a job of a single person, it takes the whole team and it takes the other professions as well’; ‘it was faster to bring it to the paper but to bring it live, to actually thresh out the solution, we needed people from different backgrounds to give their input to explain if it’s possible or not’. The role of mentors was highlighted and appreciated by all participants, though some felt they could have been more proactive: ‘I think that mentors need to be active earlier. Because by the time they started moving around and everything, teams had already started looking very deeply into the idea that they had sort of put themselves on the path’.
We also questioned them regarding the process and organisation of the event and received mixed reviews. Some were impressed by time management, while others were of the opinion that it could have been more streamlined.
Focused group discussion: ideas in incubation
Eleven months after the hackathon, the two original facilitators (KG and SS) conducted another hour-long FGD with participants whose projects were selected for incubation. The conversation revolved around enabling and disabling factors that the participants faced. Themes that surfaced over discussion included team dynamics and challenges and processes encountered.
Diversity and hierarchy within the team were identified as important contributors to a team’s success. For instance, teams that were successful noted that the presence of an IT professional helped them expedite the process of developing software for their apps. One participant pointed out that the presence of a business major would have helped his project pick up momentum early on in its course; instead, he was left to learn a multitude of tasks on his own. Multidisciplinary teams were deemed helpful by all participants, and some felt that ‘diversity on levels among the team members also helped’.
A lack of hierarchy within teams reportedly caused a lot of problems, with CEOs at the same level of the rest of the team finding it harder to be motivating forces. A few felt more strongly about scheduling, since people like residents were much harder to follow-up with, causing them to drop out. One team that had participants from outside of AKU managed to uphold the team structure by forming a core group that met more often and consisted of persons working at AKU, and another extended group that met less frequently but was onboard. One hacker felt that a new learning curve for most tasks meant that team members found it difficult to complete, and their busy schedules in healthcare just made quitting the project an easier option.
According to one participant, absence of a mentor personally invested in the project was a big challenge, since at one point their mentor ‘lost interest in the project’ which in part led them to leave midway. Conversely, another participant referred to their mentor as someone who was actively ‘mentoring, advising, taking suggestions, organizing meetings and inviting people who could help us develop this project…’ and that he/she was ‘very important in keeping us together’. Similarly, another hacker found their mentor’s contributions crucial as he/she provided them with a ‘clear milestone plan’. One participant cited difficulty with the ‘communication to convince’, that he/she felt was required at every step, in order to convey ideas to team members, judges and various potential stakeholders. Others also felt that the process of converting an idea into a prototype, and then taking it to the market, was swamped with multiple to-do’s and a lot of going back and forth. In the words of a hacker, ‘It has given me a sense of what it really takes to do something innovative, how many steps are involved, how many stakeholders are considered and a lot of things from the business, financial and marketing perspectives; things we are not used to doing as clinicians and medical educators’.
Discussion
Medical education is deeply embedded within the healthcare system, and the education of medical students and postgraduates has to be tied into clinical practice to allow exposure to patients. Since healthcare is a high-stakes service, the development of the curriculum has to be focused and highly regulated, with innovation of any sort having to go through many different checkpoints before being implemented.10
Prior interventions have shown how medical educators benefit from the insight of educators from different fields.11 Furthermore, with the growing intersection of Artificial Intelligence (AI), data science and technology in developing medical solutions, as well as the increasing interest in global health, the future of medical practice is projected to be cross-disciplinary and ambulatory, a stark contrast to its current institutionalised incarnation.12 For innovation in the delivery of healthcare education, hackathons may just be the catalyst required to target and revamp a multitude of conventional practices that remain upheld rigorously in many parts of the world, but especially in low-income to middle-income countries (LMICs).13
Our cohort of participants demonstrated inadequate representation and participation from non-medical professionals—with less than 10 individuals from IT and Engineering combined. This resulted in a dearth of diversity, and teams suffered throughout their progression because members had to attend to these aspects of their prototypes on their own. Professionals from humanities, arts and sciences did not participate and only one business professional was present. This highlights the need for improving recruitment of participants from non-medical fields, something that may be achieved with better marketing.
As the hackathon allowed teams access to the in-house incubator easily, the bootstrapping stage of a startup was circumvented and the selected teams found themselves in the seed stage, which involves teamwork and entry into market.14 Unavailability or clashing schedules of different members of the team led them to drop out as goals became progressively difficult to reach and commit to on their own. Lack of clear communication regarding roles and responsibilities, especially in teams consisting of students only, were mostly due to absence of an authoritative figure – which we consider to be a significant factor resulting in dropout from the incubation programme.15
Across existent literature, hackathons provide a semi-formal setting where professionals and students from different backgrounds, in coming together, can use this opportunity to effectively break stereotypical notions and practices regarding hierarchy in the healthcare set up. Surprisingly, teams that had members on different levels in their careers (ie, attending doctors, residents and students) fared better and were more likely to stick together as a team during the posthackathon phase.15 This goes to show that while innovation-centric endeavours may benefit from a lack of hierarchy, the incubation cycle, with its focus on achieving set milestones and producing tangible results, fares better with defined leadership and structure.
The one major finding across all teams was the need for multiple revisions of a project after beginning incubation. A clearer understanding of the process of incubation by the incubates could help prevent this. On the side of the incubator, as the teams were mostly homogenous in their competencies, a mentorship style focusing on positive feedback could have provided better results.16 The projects should also undergo significant scrutiny in the initial hours of the hackathon so that subsequent changes need not drastically alter the course of the project, requiring more groundwork from scratch. This concept was well captured in a recent paper that explored an extended hackathon framework and curriculum as an educational module to promote medical innovation.17
While the hackathon model has been accredited with expediting innovation in multiple settings, logistical hurdles and financial restrictions have to be overcome in LMICs in propelling the ideas beyond incubation.18 An encouraging finding of our study was the adequate gender representation; in fact our hackathons at AKU have consistently shown over-representation of young women as hackers; therefore, these events can be a promising opportunity for women entering the innovation and entrepreneurial work force. The favourable gender distribution for women is different from what is generally seen at hackathons and incubation programmes in the West—with males representing the predominant gender (unpublished data; Anver MT, Mehmood F, Siddiqui AO, Khalid R, Mian A. Hackathon: An Innovative Approach towards Organizational Improvement in Pakistan. 2019). An overwhelming majority of our participants also appear to be young. This is consistent with findings from other hackathons organised globally,19 where the younger generation seems to be at the forefront in trying to develop technological solutions to identified problems.
As far as shortcomings of this study are concerned, we were limited to the participants and our conversations with them for our assessments and analyses. A postevent FGD at 6 months and then at 1 year would have resulted in a better understanding of the progress of the teams, instead of one snapshot at 11 months.
In terms of future directions, it would be exciting to develop pretest and post-test strategies to prove a causal relationship between interventions such as hackathons and teaching and learning outcomes such as skills acquired, retention of knowledge and so on.
Conclusion
Hackathons provide invaluable opportunities for cross-disciplinary learning and exchange of ideas, as we have observed consistently in all the hackathons that we have conducted to date. Diversity among participants is an essential characteristic, which also helps them counter challenges more robustly throughout the business incubation process. Despite a difference in culture, the outcomes of a hackathon can be achieved if participants are well oriented, mentored and provided structured feedback.
References
Footnotes
Contributors WAB has written the discussion and formatted the article for this journal. He has also incorporated all changes from the review. WHF provided references and wrote parts of the methodology section. KG and SS both designed and executed the interventions mentioned in the research paper. QS compiled the results and wrote the results section. AM is the principle Investigator and has supervised the composition and progression of this paper across all stages.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient consent for publication Not required.
Ethics approval Ethical approval was obtained from the Aga Khan University’s Ethical Review Committee (ERC #9532389) prior to starting of the study. Consent for participation in the hackathon and subsequent incubation programme was obtained via signed written document.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement Data are available on reasonable request. All data relevant to the study are included in the article or uploaded as supplemental information.