Ohno Yukiko, Kato Riri, Ishikawa Haruki, Nishiyama Tomohiro, Isawa Minae, Mochizuki Mayumi, Aramaki Eiji, Aomori Tohru
Faculty of Pharmacy, Keio University, Tokyo, Japan.
Nara Institute of Science and Technology, Nara, Japan.
JMIR Form Res. 2024 Jun 4;8:e55798. doi: 10.2196/55798.
Large language models have propelled recent advances in artificial intelligence technology, facilitating the extraction of medical information from unstructured data such as medical records. Although named entity recognition (NER) is used to extract data from physicians' records, it has yet to be widely applied to pharmaceutical care records.
In this study, we aimed to investigate the feasibility of automatic extraction of the information regarding patients' diseases and symptoms from pharmaceutical care records. The verification was performed using Medical Named Entity Recognition-Japanese (MedNER-J), a Japanese disease-extraction system designed for physicians' records.
MedNER-J was applied to subjective, objective, assessment, and plan data from the care records of 49 patients who received cefazolin sodium injection at Keio University Hospital between April 2018 and March 2019. The performance of MedNER-J was evaluated in terms of precision, recall, and F-score.
The F-scores of NER for subjective, objective, assessment, and plan data were 0.46, 0.70, 0.76, and 0.35, respectively. In NER and positive-negative classification, the F-scores were 0.28, 0.39, 0.64, and 0.077, respectively. The F-scores of NER for objective (0.70) and assessment data (0.76) were higher than those for subjective and plan data, which supported the superiority of NER performance for objective and assessment data. This might be because objective and assessment data contained many technical terms, similar to the training data for MedNER-J. Meanwhile, the F-score of NER and positive-negative classification was high for assessment data alone (F-score=0.64), which was attributed to the similarity of its description format and contents to those of the training data.
MedNER-J successfully read pharmaceutical care records and showed the best performance for assessment data. However, challenges remain in analyzing records other than assessment data. Therefore, it will be necessary to reinforce the training data for subjective data in order to apply the system to pharmaceutical care records.
大语言模型推动了近期人工智能技术的进步，有助于从诸如病历等非结构化数据中提取医学信息。尽管命名实体识别（NER）用于从医生记录中提取数据，但它尚未广泛应用于药学服务记录。
在本研究中，我们旨在探讨从药学服务记录中自动提取患者疾病和症状信息的可行性。使用医学命名实体识别 - 日语（MedNER - J）进行验证，这是一个为医生记录设计的日语疾病提取系统。
将MedNER - J应用于2018年4月至2019年3月在庆应义塾大学医院接受头孢唑林钠注射的49例患者的护理记录中的主观、客观、评估和计划数据。从精确率、召回率和F值方面评估MedNER - J的性能。
主观、客观、评估和计划数据的NER的F值分别为0.46、0.70、0.76和0.35。在NER和正负分类中，F值分别为0.28、0.39、0.64和0.077。客观数据（0.70）和评估数据（0.76）的NER的F值高于主观和计划数据，这支持了客观和评估数据的NER性能优势。这可能是因为客观和评估数据包含许多技术术语，类似于MedNER - J的训练数据。同时，仅评估数据的NER和正负分类的F值较高（F值 = 0.64），这归因于其描述格式和内容与训练数据的相似性。
MedNER - J成功读取了药学服务记录，并且在评估数据方面表现最佳。然而，在分析评估数据以外的记录时仍存在挑战。因此，为了将该系统应用于药学服务记录，有必要加强主观数据的训练数据。
