Taseh Atta, Sasanfar Souri, Chan Michelle, Sirls Evan, Nazarian Ara, Batmanghelich Kayhan, Bean Jonathan F, Ashkani-Esfahani Soheil
Foot & Ankle Research and Innovations Laboratory (FARIL), Department of Orthopaedic Surgery, Mass General Brigham, Harvard Medical School, 158 Boston Post Road, Weston, MA, 02493, United States, 1 7818279613.
Musculoskeletal Translational Innovation Initiative, Carl J. Shapiro Department of Orthopaedic Surgery, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, United States.
JMIR Med Inform. 2025 Jul 14;13:e66973. doi: 10.2196/66973.
Standardized registries, such as the International Classification of Diseases (ICD) codes, are commonly built using administrative codes assigned to patient encounters. However, patients with fall injury are often coded using subsequent injury codes, such as hip fractures. This necessitates manual screening to ensure the accuracy of data registries.
This study aimed to automate the extraction of fall incidents and mechanisms using natural language processing (NLP) and compare this approach with the ICD method.
Clinical notes for patients with fall-induced hip fractures were retrospectively reviewed by medical experts. Fall incidences were detected, annotated, and classified among patients who had a fall-induced hip fracture (case group). The control group included patients with hip fractures without any evidence of falls. NLP models were developed using the annotated notes of the study groups to fulfill two separate tasks: fall occurrence detection and fall mechanism classification. The performances of the models were compared using accuracy, sensitivity, specificity, positive predictive value, negative predictive value, F1-score, and area under the receiver operating characteristic curve.
A total of 1769 clinical notes were included in the final analysis for the fall occurrence task, and 783 clinical notes were analyzed for the fall mechanism classification task. The highest F1-score using NLP for fall occurrence was 0.97 (specificity=0.96; sensitivity=0.97), and for fall mechanism classification was 0.61 (specificity=0.56; sensitivity=0.62). Natural language processing could detect up to 98% of the fall occurrences and 65% of the fall mechanisms accurately, compared to 26% and 12%, respectively, by ICD codes.
Our findings showed promising performance with higher accuracy of NLP algorithms compared to the conventional method for detecting fall occurrence and mechanism in developing disease registries using clinical notes. Our approach can be introduced to other registries that are based on large data and are in need of accurate annotation and classification.
标准化登记系统，如国际疾病分类（ICD）编码，通常是使用分配给患者就诊的管理代码构建的。然而，跌倒受伤患者通常使用后续的损伤代码进行编码，如髋部骨折。这就需要人工筛查以确保数据登记的准确性。
本研究旨在使用自然语言处理（NLP）自动提取跌倒事件及机制，并将该方法与ICD方法进行比较。
医学专家对跌倒导致髋部骨折患者的临床记录进行回顾性审查。在跌倒导致髋部骨折的患者（病例组）中检测、标注并分类跌倒发生率。对照组包括无任何跌倒证据的髋部骨折患者。利用研究组的标注记录开发NLP模型，以完成两项独立任务：跌倒发生检测和跌倒机制分类。使用准确率、灵敏度、特异度、阳性预测值、阴性预测值、F1分数和受试者工作特征曲线下面积比较模型性能。
最终分析纳入了1769份用于跌倒发生任务的临床记录，783份用于跌倒机制分类任务的临床记录。使用NLP进行跌倒发生检测的最高F1分数为0.97（特异度=0.96；灵敏度=0.97），跌倒机制分类为0.61（特异度=0.56；灵敏度=0.62）。与ICD编码分别为26%和12%相比，自然语言处理能够准确检测高达98%的跌倒发生率和65%的跌倒机制。
我们的研究结果显示，在使用临床记录开发疾病登记系统时，与传统方法相比，NLP算法在检测跌倒发生和机制方面具有更高的准确性，表现出良好的性能。我们的方法可引入到其他基于大数据且需要准确标注和分类的登记系统中。
