Jump to comment:

• Faulty Methodologies for Testing Technological Solutions for Language Barriers Risk Patient Safety
  Jonathan Downie
  Published on: 22 April 2024

• Published on: 22 April 2024

  Faulty Methodologies for Testing Technological Solutions for Language Barriers Risk Patient Safety

  • Jonathan Downie, Consultant Interpreter Independent Researcher

  Dear Editor,

  It is disappointing at best to read that the authors feel it appropriate to say that "[Google Translate] has the potential to routinely facilitate effective one-way oral communication between English-speaking physicians and Spanish-speaking patients with limited English proficiency," despite their own results showing accuracy reducing as the number of sentences increased. These results were themselves not derived from a robust methodology.

  Researchers in Machine Translation have pointed out that flaws in sentence-level evaluations for some time (Läubli et al 2018) and the use of a single rater runs counter to known research on inter-rater reliability in the assessment of interpreting quality (Han 2015, Wang et al 2015). In addition, scoring a technological solution and suggesting future uses on the basis of a single dataset in only one language direction, under conditions that are not realistic, demonstrates a lack of familiarity with longstanding field research on the importance of interaction in medical interpreting (Wadensjö 1992, Clifford 2004, Baraldi and Gavioli 2015).

  In light of the above evidence, it would seem that the use of Google Translate or any similar technology should not be suggested under any clinically important circumstances until such a time as each individual solution has been through robust testing, including field trials with patient feedback and tracking of patient outcomes. Any use of automated speech...

  Show More

  Dear Editor,

  It is disappointing at best to read that the authors feel it appropriate to say that "[Google Translate] has the potential to routinely facilitate effective one-way oral communication between English-speaking physicians and Spanish-speaking patients with limited English proficiency," despite their own results showing accuracy reducing as the number of sentences increased. These results were themselves not derived from a robust methodology.

  Researchers in Machine Translation have pointed out that flaws in sentence-level evaluations for some time (Läubli et al 2018) and the use of a single rater runs counter to known research on inter-rater reliability in the assessment of interpreting quality (Han 2015, Wang et al 2015). In addition, scoring a technological solution and suggesting future uses on the basis of a single dataset in only one language direction, under conditions that are not realistic, demonstrates a lack of familiarity with longstanding field research on the importance of interaction in medical interpreting (Wadensjö 1992, Clifford 2004, Baraldi and Gavioli 2015).

  In light of the above evidence, it would seem that the use of Google Translate or any similar technology should not be suggested under any clinically important circumstances until such a time as each individual solution has been through robust testing, including field trials with patient feedback and tracking of patient outcomes. Any use of automated speech translation predicated on immediate cost savings should also take into account the fact that effect professional human interpreting is already known to significantly reduce overall healthcare costs (Flores 2005, Jacobs et al 2007).

  Show Less

  Conflict of Interest:
  While the author is a consultant interpreter and interpreting researcher, his professional work does not include medical interpreting.

  • Back to top