Article Text

Download PDFPDF
Original research
Comparing ‘Twitter’ polls results with an online survey on surgeons perspectives for the treatment of rectal cancer
  1. Antonio Caycedo-Marulanda1,
  2. Sunil V Patel1,
  3. Chris P Verschoor2,
  4. Sami A Chadi3,
  5. Gabriela Möslein4,
  6. Manoj Raval5,
  7. Amy Lightner6,
  8. Manish Chand7,
  9. Rosa Jimenez-Rodriguez8,
  10. Joep Knol9,
  11. Yasuko Maeda10,
  12. John R T Monson11,
  13. Steven D Wexner12,
  14. Julio Mayol13
  1. 1 Surgery, Queen's University, Kingston, Ontario, Canada
  2. 2 Research Institute, Health Sciences North, Sudbury, Ontario, Canada
  3. 3 Surgery, University Health Network, Toronto, Ontario, Canada
  4. 4 Surgery, University Witten Herdecke Faculty of Health, Witten, Nordrhein-Westfalen, Germany
  5. 5 Surgery, The University of British Columbia, Vancouver, British Columbia, Canada
  6. 6 Surgery, Cleveland Clinic, Cleveland, Ohio, USA
  7. 7 Surgery, University College London, London, London, UK
  8. 8 Surgery, Hospital Universitario Virgen del Rocio, Sevilla, Andalucía, Spain
  9. 9 Department of Surgery, Jessa Ziekenhuis Campus Virga Jesse, Hasselt, Limburg, Belgium
  10. 10 Surgery, Western General Hospital, Edinburgh, Edinburgh, UK
  11. 11 Surgery, AdventHealth Central Florida, Orlando, Florida, USA
  12. 12 Colorectal Surgery, Cleveland Clinic Florida, Weston, Florida, USA
  13. 13 Surgery, Hospital Clinico Universitario San Carlos, Madrid, Madrid, Spain
  1. Correspondence to Dr Antonio Caycedo-Marulanda, Surgery, Queen's University, Kingston, ON K7L 2V7, Canada; Antonio.Caycedo{at}Kingstonhsc.ca

Abstract

Introduction Traditional surveys (including phone, mail and online) can be valuable tools to obtain information from specific communities. Social media apps such as Twitter are being increasingly adopted for knowledge dissemination and research purposes. Twitter polls are a unique feature which allows for a rapid response to questions posed. Nonetheless Twitter does not constitute a validated survey technique. The objective was to compare the similarities of Twitter polls in describing practice patterns for the treatment of rectal cancer.

Methods A survey on the management of rectal cancer was designed using modified Delphi methodology. Surgeons were contacted through major colorectal societies to participate in an online survey. The same set of questions were periodically posted by influencers on Twitter polls and the results were compared.

Results A total of 753 surgeons participated in the online survey. Individual participation in Twitter ranged from 162 to 463 responses. There was good and moderate agreement between the two methods for the most popular choice (9/10) and the least popular choice (5/10), respectively.

Discussion It is possible that in the future polls available via social media can provide a low-cost alternative and an efficient, yet pragmatic method to describe clinical practice patterns. This is the first study comparing Twitter polls with a traditional survey method in medical research.

Conclusions There is viable opportunity to enhance the performance of research through social media, however, significant refinement is required. These results can potentially be transferable to other areas of medicine.

  • colorectal surgey
  • health
  • delivery of health care

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Introduction

Traditionally, healthcare research surveys have been conducted through face-to-face interviews, paper-based questionnaires or telephone interviews. Previous studies have suggested these methods can be considered equivalent, with paper-based surveys being less costly than in person interviews.1 Survey data have inherent limitations due to their methodology, including indirect measure of attitudes and behaviours, especially when self-reporting occurs.2 In addition, survey data are limited by low response rates, delayed results and even difficulty in accessing a desired population. Surveys still remain an important method to gather valuable data that may be unobtainable using other methods.

With the increasing use of hand-held devices and personal computers, online platforms have provided an alternative method to access and survey a preidentified population. This approach has the potential to be faster, less expensive and perhaps more accurate than traditional designs. Additionally, newer platforms offer a facile process of data collection, decrease cost, flexible and easy customisation, and the ability to obtain partial vs total analysis as an integrated feature. At present, there is a multitude of online services available to develop and distribute surveys, most are free or inexpensive to use. Currently, universities and healthcare institutions access those platforms or have developed their own for research purposes.

Along with digitalisation various outlets have surfaced allowing for effective and rapid exchange of information. One of the most popular digital platforms among clinicians is Twitter,3 which was created in 2006. Currently, Twitter has over 330 million users around the world, generating over 500 million tweets per day. There are 68 million users in the USA, which correspond to 21% of all Twitter users, with the remaining 79% distributed across other countries. Among those many healthcare users,4 there are innumerable Twitter communities, those are commonly promoted by the use of ‘hashtags’ used to expand topics for discussion on common interests with continuous back and forth traffic of information. The platform provides any user with the ability to generate simple polls, those can include up to four options for other users to vote on. The duration of the poll can be predetermined and at the end users can see the final results. To date, there is limited evidence of the accuracy or efficacy of using social media (SoMe) to conduct medical research surveys.5 It is unknown who engages with Twitter Surveys or whether the results are reflective of the population being assessed. There are generic stats on who engages on this platform, for instance the large majority of tweets come from a small minority of tweeters, 80% of tweets in the USA are generated by 10% of users, at the same time 45% of users have at least a college degree, and 71% of Americans users have it as a news source.6

Due to Twitter’s rapid ability to gather information and its notorious popularity among doctors,4 we hypothesised that Twitter polls can provide similar and reflective results to other survey methods. Our objective was to assess this hypothesis, by evaluating responses to currently controversial components of rectal cancer care, understanding that we are more interested in the comparison than the actual answers related to the questions themselves.

Methods

In order to compare results between a traditional online survey and the Twitter polls a series of questions and clinical scenarios related to the more contentious aspects of rectal cancer care were developed. The content and wording was discussed and refined by the authors by using three rounds of the Delphi methodology. It was predetermined that only 10 questions would be included in the survey to maximise engagement and minimise participant attrition. After this process, the final content and design of the questions were agreed on by authors (online supplemental appendix A). Initial themes were discussed among the authors to identify domains of interest. These were then use to create sample questions that exceeded the number of proposed questions in order to create a sample from which the authors could identify those that best addressed key concepts for item generation. A sequential process of modification and item reduction was performed with multiple Delphi rounds and e-discussions to create the final survey.7

Supplemental material

Given the specialty focus of the authors, we decided to engage a number of colorectal surgery professional societies from multiple countries to request dissemination of an online questionnaire. The target population was colorectal surgeons from around the world. Surgical professional societies from multiple countries were asked to disseminate the online survey through their membership, including: The American Society of Colon and Rectal surgeons, The Association of Coloproctology of Great Britain and Ireland, The European Society of Coloproctology, The Canadian Society of Colon and Rectal surgeons, The Colorectal Surgical Society of Australia and New Zealand, La Sociedad Argentina de Coloproctologia, Sociedade Brasileira de Coloproctologia, La Sociedad Colombiana de Coloproctologia, La Sociedad Espanola de Coloproctologia and The Mediterranean society of Coloproctology. The accepting dates for participation were variable among societies. Authors refrained from promoting the survey on social media.

The link was open from 6 June 2019 to 16 September 2019, the closing date was decided to be 1 month after participation acceptance from the last society . Half way through, the societies were asked to remind their members to take part in the survey. To ensure there was no significant crossover, the second phase of the study was initiated 1 month after the completion of the online survey. This performed through Twitter, was initiated by posting one question at a time, on predetermined intervals of 48 hours between each question. Each question was posted by @juliomayol, who had the largest number of followers (30 000). The remaining authors, all of whom have a known social media presence (followers range of 600–12 000), subsequently retweeted each post.

Results from the survey and the Twitter polls were collated and compared. Descriptive statistics were used. Differences in proportions for each of the 10 questions between the two methods were reported. Agreement was reported based on repeatability of most and least popular choices. A more robust approach such as the Cohen’s kappa coefficient was not possible due to the unsuitability of such methodology for this project. Bar plots were used to graphically display the correlation of the preferred answers between the two platforms (figure 1). Analyses performed in the R-environment V.3.6

Figure 1

Graphical comparison of frequency of responses. Twitter versus online surveys. LAR, low anterior resection; TAMIS transanal minimally invasive surgery; TEMS, trananal endoscopy microsurgery; TME, Total Mesorfectal Excision.

Results

A total of 753 participants responded to the traditional survey with some questions being skipped. We identified responses from 60 different countries. The mean time required to complete the survey was 4 min and 8 s. There were two appreciable peaks of participation, the first one on 1 July with 177 entries and a second one on August 12 with 234 responses, both of which followed specific actions, first launching of the survey and subsequently the reminder to the societies’ membership, respectively. (figure 2).

Figure 2

Weekly trend of responses (Online suirvey). From 24th June to 16th September 2019.

Twitter entries showed a wide range of engagement between questions. The total duration of this phase was only 20 days. The maximum number of responses was generated by question # 5 (poorly differentiated low rectal tumor with apparent complete response) and the lowest was obtained from question # 2 (number of cases to determine high volume) with 463 and 162 voters, respectively. The 10 original tweets of the survey generated a total of 47 236 impressions, 1339 interactions and 2904 votes.

Fifty-two per cent of the online respondents and 49% from Twitter, considered 20 cases per year as the minimum number necessary to adequately understand management options and proficiently perform surgery on patients with rectal cancer. There were 316 votes on Twitter. For question 2, the preferred option was again similar in both platforms; regarding the number of rectal cancer cases necessary to be a high-volume centre, 43% in the online survey and 41% in the Twitter arm selected at least 40 per year. One hundred and sixty-two votes were entered via Twitter, as mentioned above, this question had the lowest participation rate on the Twitter arm.

Question 3, on which rectal cancer cases should be presented at multidisciplinary tumour board (MTB), was met with a nearly identical response for the preferred choice; 80% and 81% for the online and the Twitter surveys, respectively, felt all rectal cancer cases should be discussed at this forum. For this question 345 individuals participated via Twitter. The most popular answer choice for question 4 in both groups had 34% and 29% of voters support, representing those who use it regularly in their practice. There were a total of 188 Twitter respondents.

The next few questions represented clinical scenarios, number 5 presented a patient with a low rectal tumour with poor differentiation, and with apparent clinical response. Fifty-four per cent of the online respondents indicated they would perform a low anterior resection as originally planned, similarly to 58% of the 463 Twitter voters, with the highest number of Twitter participants as previously mentioned.

Next question depicted a case of a 65-year-old obese male with a T2N0 rectal cancer at 6 cm from the verge, requiring surgery. The options given concerned the surgical approach. The most popular choice for both arms was conventional laparoscopy, with 40% of the online survey participants and 43% of the 360 Twitter voters choosing the same answer. Number 7 was another clinical scenario, presenting a 52-year-old woman with a normal body mass index and a mid/upper rectal cancer who underwent laparoscopic surgery. We inquired whether a diverting stoma should be installed. The preferred option was equal for both arms, with 40% of the online group and 52% of the 316 Twitter participants selecting NO.

Question 8 explored the perceived distance surgeons would feel is safe for a distal margin. Answers were diverse for both groups. Interestingly 46% of the online participants considered that over a 1 cm is required whereas 29% of 248 Twitter users considered that 2 mm was the minimum distance for a safe distal margin.

The clinical scenario given in question #9 was a 59-year-old man with a posterior T3bN1 low rectal cancer, with evidence of extramural vascular invasion. The patient expressed interest in sphincteric preservation. Fifty-eight per cent of the online participants said they would perform an LAR, similar to the 52% of the 341 Twitter participants said they would also perform an LAR. The final question explored on the use of total neoadjuvant therapy (TNT) versus traditional neoadjuvant and adjuvant therapy in a 54-year-old man with a locally advanced tumour and features of aggressiveness. The online group indicated that 66% would favour TNT. The Twitter group mirrored this response with 64% of 165 users indicating usage of TNT. Agreement between the most and least popular choices for the study was 90% and 50%, respectively. A comparison of answers in both groups is presented in table 1.

Table 1

Number of responses on each platform are at the top of the question, below is the comparative proportions of each answer

Discussion

We found important similarities between a traditional online survey and Twitter polls results. Good agreement between the top choices was identified (9 of 10 questions). This is an important finding worthy of further consideration since it is not infrequent for physicians in all specialties to use Twitter and other platforms as a method to rapidly obtain and exchange information related to the management of their patients. Agreement for the least popular choices was moderate (5 of 10 questions).

Data management continues to rapidly evolve and digitalisation advances, the integration of social media outlets (SoMe) presents a myriad of opportunities, which can significantly accelerate the exchange of knowledge. Many individuals, including politicians, athletes and celebrities, have used SoMe for promotional purposes, while others use SoMe for entertainment and education.6 More recently, SoMe has turned into an important source of dissemination and exchange of information among physicians simultaneously Twitter has become increasingly popular among healthcare professionals.4

For our exercise, questions were carefully designed using Delphi methodology. We considered the content of the question needed to be relevant and interesting in order to generate engagement from the surveyed. For the primary objective of the study the questions themselves were less important than the actual correlation of the results between the two methods. We found good agreement for the most popular choice, with similar proportions despite that for the online survey a total of 753 entries were recorded. On the survey, it was certain that responses came from surgeons affiliated to well-established colorectal societies. Less agreement was identified when comparing the least preferred option for each question. If we explore the answers in detail there are discrepancies. For the most part the preferred choices are very similar except for question number 8 (Defining safe distal margin). The online group preference was for >1 cm, whereas in Twitter preferences were distributed homogeneously for all choices. For the top choice on almost all questions, we identified a strong correlation between the two methods, distribution of less popular choices was widely heterogeneous. Proportions and preferences are graphically depicted in figure 2.

Traditionally, surveying physicians has had a lower response rate than the rest of the population.7 Nonetheless it is unclear whether physicians are more or less likely to respond to a formal survey request from their medical organisation, clearly marked as for research purposes, vs what appears to be an informal posting by a surgeon who they follow on social media. It is theoretically possible that Twitter polls may become an efficient method to obtain information on clinicians regarding their preferences and medical practices. For instance, in our project Twitter users saw the posted questions more than 47 k times, which would be impossible by any traditional method. However, using impressions as a metric still represents a challenge, as it measures ‘potential views’, not actual views. Not everyone following an individual physician will be a physician and among physicians, they will not all be of the same specialty.

Postings generated discussions as some participants considered some subjects would have obvious answers, therefore, questioning the value of the question. The discrepancies in results and the immediate discussion demonstrated that no topic is beyond doubt. A good illustration is question 3 (cases to be presented at MTB) where one comment was ‘all of them, which should not even be a question’. Interestingly 20% of the participants in both arms did not agree, perhaps much more important was a real time discussion that ensued on a subject that seemed obvious and completely settled. For the purpose of our project, all the tweets and initial retweets were sent to target users who belonged to rectal cancer surgical communities and the authors are colorectal surgeons who use Twitter frequently to exchange scientific medical information.

Digital platforms can provide enhanced methods for exchange and dissemination of information and thus constitute an important facet of medical research to further explore. Twitter polls are commonly used to pose clinical or academic questions about opinions or trends on the best possible treatments or options regarding clinical scenarios or theoretic concepts. Those polls as opposed to conventional surveys can reach audiences very rapidly and generate almost immediate results. The real influence and impact of SoMe in medicine is not completely understood and it possibly never will since it continues to change and evolve in a very rapid fashion. SoMe has had a powerful impact on interpersonal interactions, such as the flattening of hierarchies by allowing direct communication among the ranks, for example, mentee and mentor; trainee and a highly reputed specialist. Additionally it has ‘eliminated’ geographical barriers in a novel fashion.

At this time, Twitter polls are limited in terms of the number of characters that can be used to post or formulate a question, 280 in total and only 4 options that can be integrated into the construction of each poll. We theorised that answers between these two methods (SurveyMonvey and Twitter) targeting colorectal surgeons with the same set of questions on both arms would not be very different, they were not identical with 90% and 50% agreement for the most and least popular choice respectively (table 1). The immediacy of the results in our Twitter poll was a clear advantage in addition to the ability to create discussion in real time. Another potential advantage is the results on Twitter can be instantly shared with the participants, which can generate further discussion.

A well-designed survey can provide powerful information that can guide future research, influence clinical practice and even steer the development and provision of wide-range programmes and clinical services.2 Collected data are as useful as they can convey information accurately and consistently. Research exploring the potential utility of SoMe for medical research continues to expand, the possibility that Twitter polls can be used as a method to reliably survey clinicians is in its infancy. A validated survey instrument becomes relevant to the questionnaire design,8 this is certainly not the case of Twitter polls today.

Currently, it is not possible to determine a response rate on Twitter polls.9 It is relevant to mention the existence of virtual communities and groups of interest within the platform10 nevertheless, there is no control on who is engaging and participating in these ‘communities’ since Twitter is open to the general public therefore anyone could participate. This can skew the results of polls, the presumption is virtual communities are self-selected and have virtual boundaries established by the predefined interest of the person or entity posting the question, however, this remains impossible to verify. Being on Twitter is by itself self-selecting as it only includes a small group of surgeons, further bias is created by belonging to a specific virtual community. Those in Twitter and social media outlets are not necessarily a representative sample of the colorectal surgery population, nonetheless a similar argument can be made regarding the official colorectal surgical societies. Furthermore, we faced the inability to establish a response rate, as we had no method to verify how many individuals were reached by either platform. Due to the limited number of questions, specific surgical practice information of the respondents was not collected.

Twitter seems a promising clinical research alternative, growing popularity, still with significant concerns about professionalism, confidentiality, potential for misinformation and inability to verify sources.4

Conclusions

Surveys remain a valuable method to obtain information from samples of specific population. Twitter is a popular SoMe platform among the medical community. Some of the advantages include the possibility of almost immediate interaction and the ‘flattening of traditional hierarchies’. This platform offers the possibility of conducting polls, and thus is frequently used by clinicians to exchange information. At the present time, Twitter polls are in no position to replace formal surveys, however, they may potentially evolve to become important tools to contribute to scientific enquiry.

References

Footnotes

  • Twitter @caycedomarula

  • Contributors AC-M, SVP and JM planned the study. AC-M, SVP, SAC, MR, MC, JK, JRTM, SDW and JM designed the questions. All authors approved the content of the survey. AC-M, SVP, CPV, SDW and JM conducted the survey. All authors were actively involved in the Twitter phase of the study. AC-M, SVP, CPV and SAC performed the central analyses. All authors participated in writing the manuscript. All authors provided edits, comments and feedback until a final version of the draft was approved. AC-M submitted the study on behalf of the all authors.

  • Funding This manuscript has been self-funded by the authors.

  • Competing interests None declared.

  • Patient consent for publication Not required.

  • Ethics approval Ethical approval was centrally obtained.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data availability statement All data relevant to the study are included in the article or uploaded as online supplemental information.