Boston Combined Residency Program in Pediatrics at Boston Children's Hospital and Boston Medical Center, Boston, Massachusetts.
Departments of Pediatrics.
Pediatrics. 2024 Jul 1;154(1). doi: 10.1542/peds.2023-065573.
Patients who speak languages other than English face barriers to equitable healthcare delivery. Machine translation systems, including emerging large language models, have the potential to expand access to translation services, but their merits and limitations in clinical practice remain poorly defined. We aimed to assess the performance of Google Translate and ChatGPT for multilingual translation of pediatric discharge instructions.
Twenty standardized discharge instructions for pediatric conditions were translated into Spanish, Brazilian Portuguese, and Haitian Creole by professional translation services, Google Translate and ChatGPT-4.0, and evaluated for adequacy (preserved information), fluency (grammatical correctness), meaning (preserved connotation), and severity (clinical harm), along with assessment of overall preference. Domain-level ratings and preferred translation source were summarized with descriptive statistics and compared with professional translations.
Google Translate and ChatGPT demonstrated similar domain-level ratings to professional translations for Spanish and Portuguese. For Haitian Creole, compared with both Google Translate and ChatGPT, professional translations demonstrated significantly greater adequacy, fluency meaning, and severity scores. ChatGPT (33.3%, P < .001) and Google Translate (23.3%, P = .024) contained more potentially clinically significant errors (severity score ≤3) for Haitian Creole than professional translations (8.3%). Professional Haitian Creole (48.3%) and Portuguese (43.3%), but not Spanish (15%), translations were most frequently preferred among translation sources.
Machine translation platforms have comparable performance to professional translations for Spanish and Portuguese but shortcomings in quality, accuracy, and preference persist for Haitian Creole. Diverse multilingual training data are needed, along with regulations ensuring safe and equitable applications of machine translation in clinical practice.
讲英语以外其他语言的患者在获得公平医疗服务方面面临障碍。机器翻译系统，包括新兴的大型语言模型，有可能扩大翻译服务的获取途径，但它们在临床实践中的优缺点仍未得到明确界定。我们旨在评估 Google Translate 和 ChatGPT 在翻译儿科出院指导方面的多语言翻译表现。
由专业翻译服务、Google Translate 和 ChatGPT-4.0 将 20 份标准化儿科疾病的出院指导翻译成西班牙语、巴西葡萄牙语和海地克里奥尔语，评估其充分性（保留信息）、流畅性（语法正确性）、意义（保留内涵）和严重程度（临床危害），以及整体偏好。用描述性统计对域级评分和首选翻译来源进行总结，并与专业翻译进行比较。
Google Translate 和 ChatGPT 在西班牙语和葡萄牙语方面与专业翻译的域级评分相似。对于海地克里奥尔语，与 Google Translate 和 ChatGPT 相比，专业翻译的充分性、流畅性、意义和严重程度评分显著更高。ChatGPT（33.3%，P<.001）和 Google Translate（23.3%，P=.024）比专业翻译（8.3%）包含更多潜在临床意义的错误（严重程度评分≤3）。专业海地克里奥尔语（48.3%）和葡萄牙语（43.3%），但不是西班牙语（15%）的翻译源在翻译源中最常被首选。
机器翻译平台在西班牙语和葡萄牙语方面的表现与专业翻译相当，但在海地克里奥尔语方面，质量、准确性和偏好方面的不足仍然存在。需要更多的多语言培训数据，以及确保机器翻译在临床实践中安全和公平应用的法规。
