• Won Lee, Elaine C Khoong, Billy Zeng, Francine Rios-Fetchko, YingYing Ma, Kirsten Liu, and Alicia Fernandez.
• University of California San Francisco, 513 Parnassus Ave, Room S-436, San Francisco, CA, 94143, USA. Won.Lee@ucsf.edu.
• J Gen Intern Med. 2023 Aug 1; 38 (10): 233323392333-2339.

BackgroundAccessing professional medical interpreters for brief, low risk exchanges can be challenging. Machine translation (MT) for verbal communication has the potential to be a useful clinical tool, but few evaluations exist.ObjectiveWe evaluated the quality of three MT applications for English-Spanish and English-Mandarin two-way interpretation of low complexity brief clinical communication compared with human interpretation.DesignAudio-taped phrases were interpreted via human and 3 MT applications. Bilingual assessors evaluated the quality of MT interpretation on four assessment categories (accuracy, fluency, meaning, and clinical risk) using 5-point Likert scales. We used a non-inferiority design with 15% inferiority margin to evaluate the quality of three MT applications with professional medical interpreters serving as gold standards.Main MeasuresProportion of interpretation exchanges deemed acceptable, defined as a composite score of 16 or greater out of 20 based on the four assessment categories.Key ResultsFor English to Spanish, the proportion of MT-interpreted phrases scored as acceptable ranged from 0.68 to 0.84, while for English to Mandarin, the range was from 0.62 to 0.76. Both Spanish/Mandarin to English MT interpretation had low acceptable scores (range 0.36 to 0.41). No MT interpretation met the non-inferiority threshold.ConclusionWhile MT interpretation was better for English to Spanish or Mandarin than the reverse, the overall quality of MT interpretation was poor for two-way clinical communication. Clinicians should advocate for easier access to professional interpretation in all clinical spaces and defer use of MT until these applications improve.© 2023. The Author(s), under exclusive licence to Society of General Internal Medicine.

9 keywords...

hide…