Article Text

PDF

Original article
Prospective, randomised controlled trial to evaluate the effect of smart glasses on vestibular examination skills
  1. Maura D Iversen1,2,3,
  2. Sheri Kiami1,
  3. Karandeep Singh4,
  4. Italo Masiello5,
  5. Johan von Heideken1,3
  1. 1Department of Physical Therapy, Movement and Rehabilitation Sciences, Northeastern University, Boston, Massachusetts, USA
  2. 2Department of Medicine, Section of Clinical Sciences Brigham & Women's Hospital, Harvard Medical School, Boston, Massachusetts, USA
  3. 3Department of Women's and Children's Health, Karolinska Institutet, Stockholm, Sweden
  4. 4Department of Medicine, Harvard Medical School, Boston, Massachusetts, USA
  5. 5Department of Clinical Science and Education, Södersjukhuset, Karolinska Institutet, Stockholm, Sweden
  1. Correspondence to Professor Maura Daly Iversen, Department of Physical Therapy, Movement and Rehabilitation Sciences, Northeastern University, 360 Huntington Avenue, Boston, MA 02115, USA; M.Iversen{at}neu.edu

Abstract

This proof-of-concept trial explored whether using Google Glass to augment instruction of vestibular assessment and treatment skills increased student competence. In 2015, 103 physiotherapy students with no prior vestibular examination training participated in a lab for adults with vestibular dysfunction led by experienced clinicians. Half were randomised (blocked on gender and self-efficacy) to standard lab instruction and half to the experiential lab wherein the instructor used the Google Glass video function in real time, along with verbal instruction. Students completed a pretest questionnaire to establish self-reported skill level in vestibular dysfunction assessment and treatment. Another faculty member assessed students' competence in these clinical skills after 7 days. A student evaluation of teaching was completed after the teaching session. No between-group differences existed regarding change in self-efficacy scores for vestibular assessment and treatment skills preintervention and postintervention. Students in the Google Glass group scored slightly better on the clinical skills check (median score of 19 (range 16–20) versus 18 (range 16–20); p=0.03). 40 students (77%) in the Google Glass group performed the assessment satisfactorily versus 30 (59%) (p=0.049) in the standard lab group. The majority of students evaluated the video quality as unacceptable. This novel study demonstrated that using Google Glass to live stream vestibular assessment and treatment techniques in first-person view had a significant but small impact on acquisition of these clinical skills.

Statistics from Altmetric.com

Introduction

Within the USA, neuromuscular symptoms, including vestibular dysfunction, comprise a large number of healthcare visits and costs.1 As the population ages, vestibular dysfunctions are projected to become increasingly more onerous for individuals and society.2

Vestibular clinical examination skills are complex, requiring integration of high-level psychomotor and cognitive functions. These are among the more difficult skills for doctor of physiotherapy (DPT) students to learn.3 Owing to the volume of curricular material and limited time to cover the content, increasing the time allocated to learning these skills is not a viable option. Resources are focused on improving the value on the time spent on learning clinical skills through the incorporation of novel teaching and learning methods, to enhance the assimilation and acquisition of these skills by DPT students.

According to Miller, the process of task-based learning to attain clinical competence involves four major stages. The first two stages are cognitively-based and the latter two stages are behaviourally-based. In the first stage, the student gathers information. In the second stage, the student learns to interpret and apply the information through case scenarios or presentations. The third stage involves demonstration of the skill and the final stage is reached when the student is able to integrate the skill into practice.4 ,5

Small group teaching formats are favoured compared to lecture-based instruction when teaching neuromuscular examination skills.6 Thus, many academic programmes integrate experiential learning through clinical skills labs combined with didactic instruction. However, in a clinical laboratory setting, there is room for a limited number of students to observe the teacher demonstrate the examination procedures and highlight important clinical findings. The use of Google Glass during clinical examination instruction has the potential to provide greater access of this instruction to more students and allows a first person view of the procedure as it is performed on a client.7 Thus, we proposed the integration of Google Glass into the vestibular clinical skills instruction.

Google Glass is a wearable device with a set of features that puts it into a new category of devices known as smart glasses. Smart glasses are defined as computerised communicators with a transparent screen and a video camera, worn as a pair of eyeglasses.8 The current model of Google Glass is a prototype that was intended for a mass market and not to specifically be used in medical education. Even though the manufacture of the current model of Google Glass was stopped in January, 2015, Google remains committed to the development of this product,9 and several other smart glasses are either in the market10 ,11 or will be in the coming year.12 ,13

The functions of Google Glass can be useful alone or in combination and have the potential to improve the knowledge transfer regarding practical procedures in medical education. Currently, there are limited numbers of published scientific papers evaluating the efficacy of smart glasses in medical teaching and learning.14–18 These papers provide mostly promising results; Russell et al16 found the use of Google Glass for telementoring medical students in the use of ultrasonography imaging proved effective. In another study, a standardised patient used Google Glass to record an encounter with medical students for faculty and student feedback, and a majority of the students thought Google Glass was a tool worth including in the clinical skill training programme.17

The concept of video recording events in medical clinical skill education is not novel.19 At the moment, there are other wearable systems with better video resolution, zoom function and superior design than Google Glass to record procedures.15 However, the introduction of a lightweight, portable and relatively inexpensive wearable technology that has all the functions of Google Glass is novel.

This randomised controlled trial aimed to determine whether Google Glass was effective in promoting the transfer of assessment and treatment skills for the management of adults with vestibular dysfunction among novice physiotherapy students. Specifically, we evaluated the Google Glass video camera function while an experienced clinical educator demonstrated vestibular dysfunction assessment and treatment skills from a first-person perspective and in real time. We hypothesised that the use of Google Glass would improve the learning of vestibular dysfunction assessment and treatment skills when compared to standard clinical teaching methods.

Materials and methods

Participants: One hundred and three second professional year DPT students, with no prior neuromuscular examination training, were recruited in March 2015 and enrolled from a large, private, non-profit research university. All students agreed to participate. These data were originally generated as a part of a classroom project that was conducted for non-research purposes. The university IRB approval was given for analysis of this data set. These data were de-identified and no prospective consent was involved.

Students completed a pretest questionnaire to establish their self-reported confidence in the use of vestibular dysfunction assessment and treatment skills using a modified version of the Acute Care Confidence survey (score 100–1000, 10 questions each scored 10–100: very uncertain represents 10, and very certain is represented by 100) developed by Greenwood and Iversen.20 All students participated in a 160 min session covering the assessment and treatment of adults with vestibular dysfunction. Two professors with more than 10 years of teaching experience in physiotherapy programmes led the clinical examination skills laboratory along with three experienced DPT clinicians. Students were divided into two groups, treatment and control. These two groups were further subdivided to accommodate classroom teaching ratios and space, resulting in 50 students per allocation group. The learning objectives were explained in detail at the beginning of the session and in the course syllabus.

Randomisation: Using a randomisation plan generator, students were block randomised based on gender and self-efficacy score to either:

  • Group A (control group): Traditional clinical skills classroom instruction that included preparatory readings and lecture, demonstration by the faculty member and practice of each of these clinical skills with fellow DPT students.

  • Group B (intervention group): Traditional clinical skills classroom instruction that included preparatory readings and lecture, demonstration by the faculty member who was wearing Google Glass during the performance of the clinical skills and practice of each of these clinical skills with fellow DPT students (figure 1). The video from the Google Glass was projected in real time on a large screen to allow students insight into what the teacher was seeing while the teacher was speaking. Prior to the trial, the DPT faculty member leading the intervention groups received training in the use of Google Glass until competency with the device was achieved.

Figure 1

Photo illustrating physiotherapy faculty member who was wearing Google Glass while demonstrating; and practising with fellow doctor of physiotherapy students.

The two clinical faculty members providing instruction in vestibular assessment and treatment skills each had more than 15 years of experience in neurorehabilitation and more than 10 years of experience teaching in a DPT programme. Both members worked together in the course to provide standardised teaching of all neuromuscular skills. One instructor (SK) led two groups of approximately 25 students with use of Google Glass, one group before and the other group after lunch, on the same day. The other instructor simultaneously led two groups of students in the same lab skills without using smart glasses. Thus, the experimental design ensured students allocated to the intervention groups received similar instruction to those in the control group.

Measurement tool: Seven days after initial instruction, another faculty member assessed students' proper use and demonstration of vestibular dysfunction clinical skills, using a standard clinical skills checklist. For the skill checklist, students were asked to demonstrate one assessment (Dynamic visual acuity, Head thrust, Head shake, Vestibular Ocular Reflex cancellation or Vergence) and one treatment manoeuvre (Hallpike-Dix, Canalith Repositioning Treatment, Brandt-Daroff habituation or Liberatory manoeuvre).21 Students were graded on the explanation and instructions provided to the mock patient, execution and handling skills, therapist position, test interpretation and proper application. The skills test of assessment and treatment manoeuvres, were scored from 0 to 20 points, with higher scores indicating better performance. A score of 15 or more was required to pass the test. The 7-day delay was implemented to eliminate students' ‘immediate recall’ bias. Students were not timetabled nor permitted to have any additional neuromuscular examination training during this 7-day period. A student evaluation of teaching survey was also administered after the teaching session and consisted of 11 questions, each scored on a Likert scale from 1 to 5, where 1 correlated with strongly disagree and 5 with strongly agree.

Outcomes: The primary outcome was the clinical skills checklist score. The secondary outcome was the student evaluation of teaching.

Sample size estimation: A mean difference in clinical skills checklist scores of 5% was assumed to be of educational significance. The test has been used for 5 years to evaluate more than 100 students per year. The normally distributed data from the prior year had a group mean score (17.8) and SD (2.02). These data were used to establish the sample size of 32 students per group with a significance level of 0.05 and a power of 80%.

Statistical methods: Descriptive statistics characterised by the sample and Wilks Shapiro tests were used to assess normality of outcome variables. Data from the clinical skills checklist form were assessed for normality and used to stratified students into two groups, those who satisfactorily performed the skills versus those who needed improvement or performed unsatisfactorily. Inferential statistics were used to confirm the effectiveness of the randomisation.

Mann-Whitney U tests were used to compare the homogeneity between the groups regarding student clinical skills checklist total scores, change in self-efficacy scores and students' evaluation of teaching questionnaire. Specific performance skills were dichotomised as needing improvement or satisfactory, and performance between groups was assessed using Pearson's χ2 tests. The established level of significant was a p value of less than 0.05.

A 5% difference in mean scores on the total psychomotor skills check scores between the groups was predetermined to be academically relevant, as that value can discriminate between traditionally accepted letter grades (eg, 90%=A-, 95%=A). Statistical analysis was performed with the statistical program SPSS for Mac V.22.0 (SPSS Inc, Chicago, Illinois, USA, http://www.spss.com).

Results

The study involved the entire cohort of 103 professional year DPT students: 51 students in the control group and 52 students in the Google Glass group. Seventy-five students in the study were Caucasian (73%), 7 (7%) were Asian, 3 (3%) were African-American, 3 (3%) were Hispanic or Latino, 5 (5%) reported being of two or more races, and 10 (10%) did not provide their race and ethnicity. The flow of the students through the study is illustrated in figure 2. Student demographics were similar for both groups, including results on the presurvey scores for self-efficacy in vestibular dysfunction examination and intervention skills (table 1). Even though 21 (20%) were post baccalaureate students, none of the students had prior neuromuscular examination training.

Table 1

Demographic features of doctor of physiotherapy students participating in the study (n=103)

Figure 2

Students flow through the study.

All students successfully completed the 160 min session of vestibular dysfunction assessment and treatment skills. Students in the Google Glass group scored better on the clinical skills check, with a median score of 19 (range 16–20) versus 18 (range 16–20); p=0.03). There was a 5.6% difference in scores between the groups. When examining the different components of the skills checklist form, we found no statistical difference between the groups except in the category assessing execution and handling of the patients during clinical assessment. Forty students (77%) in the Google Glass group performed the specified assessment satisfactorily versus 30 (59%) (p=0.049) who performed the assessment satisfactorily in the standard lab group. Regarding the execution and handling of the treatment manoeuvre, we found 19 (37%) in the Google Glass group performed the specified manoeuvre satisfactorily in every section of the test versus 16 students (31%) who performed the manoeuvre satisfactorily in the standard lab group (p=0.58).

Seventy-nine students, 39 (75%) in the Google Glass group and 40 (78%) in the control group, completed the evaluation of faculty teaching assessment. Students in the Google Glass group answered additional questions regarding the use of Google Glass. Data regarding the evaluation of teaching were similar for both groups, with a median score of 49 (range=37–55) in the Google Glass group and median score of 50 (range=38–55) in the control group (p=0.79). Twenty-six (67%) students in the Google Glass group evaluated the video quality as not acceptable, defined as a score of 2 or below on the Likert scale. Twenty-three students (59%) reported that the use of Google Glass did not enhance their learning (figure 3). Eighty-seven students, 42 (81%) in the Google Glass group and 45 (88%) in the control group, completed the preself-efficacy survey. All 103 students completed the postself-efficacy test. Thus, the difference in self-efficacy scores before and after intervention was calculated for 87 students. There was no statistically significant difference between the groups regarding change in self-efficacy, median 570 (range=100–830) in the Google Glass group versus 540 (range=70–780) in the control group (p=0.57; table 2).

Table 2

Mean differences in self-efficacy before and after intervention, clinical skills check and evaluation of teaching among physiotherapy students following the intervention and students evolution of teaching questionnaire (n=103)

Figure 3

Student evaluation of Google Glass.

Technical issues with Google Glass

We experienced several practical issues with Google Glass before and during the lab sessions. The glasses heated up to a level that prevented them from functioning properly during class. They also lost battery power after an hour. Also, initial Wi-Fi connectivity loss hindered the live streaming capability in the beginning of the class. Synchronised images projected in real time with 8–10 s delays. Additionally, we encountered problems with the image projected on the screen rotated 90° to the right, which could only be resolved by connecting to a projector mounted on its side.

Discussion

In this study, we evaluated the use of the Google Glass video camera function while an experienced clinical educator demonstrated vestibular dysfunction assessment and treatment skills from a first-person perspective. The advantage of this method compared to traditional video recording is that the video was projected in real time onto a large screen to allow students insight into what the teacher was seeing while performing the clinical skills; and the clinical procedure was described. To the best of our knowledge, this is the only study evaluating the function of the Google Glass video camera in this manner.

Knowledge regarding clinical examination skills is difficult to transfer to another person by means of writing down directions or verbalising them. The key to acquiring this type of knowledge is experience, since without some form of shared experience it is hard for the teacher and the student to share each other's thinking processes. The traditional teaching method is for students to observe how the procedure is performed and practise the skill under the supervision of the teacher. As such, smart glasses have the potential to provide a new dimension to teaching and learning this type of tacit knowledge. The benefit of smart glasses as compared to traditional video recording is that the student can access the teacher's field of vision, receiving a new perspective. We hypothesised that using Google Glass would improve efficiency of performing these new examination and treatment skills.

Data from this trial suggest the use of Google Glass may improve the acquisition and performance of vestibular assessment and treatment techniques to a small degree. However, the use of Google Glass did not appear to enhance students' self-efficacy for performing vestibular dysfunction assessment and treatment skills more than traditional laboratory instruction. One potential reason for this small impact of the Google Glass in teaching may be related to current limitations of the Google Glass technology.15 ,22 ,23 On average, students in the Google Glass group did not rate the video quality as being high. Additionally, as the instructors are highly experienced in this area and worked to develop the lab sessions, there may have been a ceiling effect, as noted by the score distribution, leading to a lack of ability to assess differences between the groups.

This study did not assess smart glasses in a clinical teaching setting, due to concern regarding patient privacy. Instead, we used the traditional teaching laboratory experience, as it was more suitable to evaluate the efficiency of smart glasses in education. Some important research questions that need to be answered regarding the use of smart glasses in education include: in which situations do smart glasses contribute to the learning? How and when is it best to measure the learning outcome? To what extent can the results be attributed to the introduction of smart glasses?

We recognise several potential study limitations. First, we used healthy faculty members to serve as patients during the demonstrations and practice sessions instead of using patients with vestibular dysfunction symptoms. This modelling of patient symptoms might have influenced our results, since the video could not identify subtle clinical findings. Second, we did not compare the Google Glass with other existing and potentially more technologically advanced suitable smart glasses or wearable camcorders. Third, there is the possibility of attrition bias, due to the rate of incomplete outcome data. However, the loss to follow-up was relatively equivalent in both groups so it should not alter the results. Fourth, the results from this study may not be generalisable to other groups of students, for example, nurses and medical students. Finally, we tested the students' competence in an assessment, but this might not reflect their competence in real practice.

Despite these limitations, our study design minimises the possibility that observed association is due to confounding.

Future versions of smart glasses will probably be specifically developed for healthcare applications, and will possibly address the hardware and software issues. Future applications of the video camera function in smart glasses could include: allowing the patient to wear the smart glasses to enable students to view themselves through the patients' eyes, thereby providing a unique patient perspective,17 or to have students stream their physical examination of a patient to a senior clinician, who can then provide real-time feedback.16 ,22 ,24

In conclusion, there were no differences between groups in self-efficacy scores at baseline or following the skills lab. Psychomotor skills and clinical decision-making regarding vestibular dysfunction increased more among the group using the Google Glass. The change in skill scores between the groups was statistically significant and this degree of change may yield a small but meaningful improvement in academic performance. Further studies are required to determine effectiveness of smart glasses to improve student outcomes with attainment of neuromuscular examination skills. While there are many potentially important applications of this new technology in medical education, new product releases may address some of the initial limitations. However, due to the increasing role of technology in facilitating the communication of medical information, it seems inevitable that smart glasses will have a role to play in medical education at some point in the future.

Acknowledgments

The team would like to thank Mr Kiami for his technical assistance with the Google Glass.

References

View Abstract

Footnotes

  • Contributors MDI was principal investigator, participated in study design and data analysis, and was involved in the writing of the manuscript. SK participated in study design and conducted the test for the study, and was involved in the writing of the manuscript. KS participated in study design and provided technical assistance, and was involved in manuscript editing. IM participated in study design and manuscript editing. JvH had primary responsibility for study design and data analysis, and was involved in the writing the manuscript. All the authors read and approved the final manuscript.

  • Funding Partial funding provided by Northeastern University and Hirsch Fellowship by Karolinska Institutet.

  • Competing interests None declared.

  • Ethics approval This study was approved by the Institutional Review Board of Northeastern University, Boston, Massachusetts, USA.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data sharing statement We are happy to share our anonymised data.

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.