Department of Information Medicine, Asan Medical Center, Seoul, Republic of Korea.
Big Data Research Center, Asan Institute for Life Sciences, Asan Medical Center, Seoul, Republic of Korea.
JMIR Med Inform. 2024 Nov 18;12:e49724. doi: 10.2196/49724.
Transformer-based language models have shown great potential to revolutionize health care by advancing clinical decision support, patient interaction, and disease prediction. However, despite their rapid development, the implementation of transformer-based language models in health care settings remains limited. This is partly due to the lack of a comprehensive review, which hinders a systematic understanding of their applications and limitations. Without clear guidelines and consolidated information, both researchers and physicians face difficulties in using these models effectively, resulting in inefficient research efforts and slow integration into clinical workflows.
This scoping review addresses this gap by examining studies on medical transformer-based language models and categorizing them into 6 tasks: dialogue generation, question answering, summarization, text classification, sentiment analysis, and named entity recognition.
We conducted a scoping review following the Cochrane scoping review protocol. A comprehensive literature search was performed across databases, including Google Scholar and PubMed, covering publications from January 2017 to September 2024. Studies involving transformer-derived models in medical tasks were included. Data were categorized into 6 key tasks.
Our key findings revealed both advancements and critical challenges in applying transformer-based models to health care tasks. For example, models like MedPIR involving dialogue generation show promise but face privacy and ethical concerns, while question-answering models like BioBERT improve accuracy but struggle with the complexity of medical terminology. The BioBERTSum summarization model aids clinicians by condensing medical texts but needs better handling of long sequences.
This review attempted to provide a consolidated understanding of the role of transformer-based language models in health care and to guide future research directions. By addressing current challenges and exploring the potential for real-world applications, we envision significant improvements in health care informatics. Addressing the identified challenges and implementing proposed solutions can enable transformer-based language models to significantly improve health care delivery and patient outcomes. Our review provides valuable insights for future research and practical applications, setting the stage for transformative advancements in medical informatics.
基于转换器的语言模型通过推进临床决策支持、患者交互和疾病预测，显示出彻底改变医疗保健的巨大潜力。然而，尽管它们发展迅速，但基于转换器的语言模型在医疗保健环境中的实施仍然有限。这部分是由于缺乏全面的审查，这阻碍了对其应用和局限性的系统理解。没有明确的指导方针和综合信息，研究人员和医生都难以有效地使用这些模型，导致研究工作效率低下，并且难以将其整合到临床工作流程中。
本范围综述通过检查基于医学的转换器语言模型的研究并将其分为 6 个任务：对话生成、问答、总结、文本分类、情感分析和命名实体识别，来解决这一差距。
我们按照 Cochrane 范围综述方案进行了范围综述。在 Google Scholar 和 PubMed 等数据库中进行了全面的文献检索，涵盖了 2017 年 1 月至 2024 年 9 月的出版物。纳入了涉及医学任务中基于转换器的模型的研究。将数据分为 6 个关键任务。
我们的主要发现揭示了将基于转换器的模型应用于医疗保健任务的进展和关键挑战。例如，涉及对话生成的 MedPIR 等模型显示出了前景，但面临隐私和伦理问题，而像 BioBERT 这样的问答模型则提高了准确性，但难以处理医学术语的复杂性。BioBERTSum 总结模型通过压缩医学文本来帮助临床医生，但需要更好地处理长序列。
本综述试图提供对基于转换器的语言模型在医疗保健中的作用的综合理解，并指导未来的研究方向。通过解决当前的挑战并探索实际应用的潜力，我们设想在医疗保健信息学方面会有显著的改进。解决已识别的挑战并实施建议的解决方案可以使基于转换器的语言模型极大地改善医疗保健服务和患者的结果。我们的综述为未来的研究和实际应用提供了有价值的见解，为医学信息学的变革性进步奠定了基础。
