Fernandes Marta B, Valizadeh Navid, Alabsi Haitham S, Quadri Syed A, Tesh Ryan A, Bucklin Abigail A, Sun Haoqi, Jain Aayushee, Brenner Laura N, Ye Elissa, Ge Wendong, Collens Sarah I, Lin Stacie, Das Sudeshna, Robbins Gregory K, Zafar Sahar F, Mukerji Shibani S, Westover M Brandon
Department of Neurology, Massachusetts General Hospital (MGH), Boston, MA, United States.
Harvard Medical School, Boston, MA, United States.
Expert Syst Appl. 2023 Mar 15;214. doi: 10.1016/j.eswa.2022.119171. Epub 2022 Nov 6.
Neurologic disability level at hospital discharge is an important outcome in many clinical research studies. Outside of clinical trials, neurologic outcomes must typically be extracted by labor intensive manual review of clinical notes in the electronic health record (EHR). To overcome this challenge, we set out to develop a natural language processing (NLP) approach that automatically reads clinical notes to determine neurologic outcomes, to make it possible to conduct larger scale neurologic outcomes studies. We obtained 7314 notes from 3632 patients hospitalized at two large Boston hospitals between January 2012 and June 2020, including discharge summaries (3485), occupational therapy (1472) and physical therapy (2357) notes. Fourteen clinical experts reviewed notes to assign scores on the Glasgow Outcome Scale (GOS) with 4 classes, namely 'good recovery', 'moderate disability', 'severe disability', and 'death' and on the Modified Rankin Scale (mRS), with 7 classes, namely 'no symptoms', 'no significant disability', 'slight disability', 'moderate disability', 'moderately severe disability', 'severe disability', and 'death'. For 428 patients' notes, 2 experts scored the cases generating interrater reliability estimates for GOS and mRS. After preprocessing and extracting features from the notes, we trained a multiclass logistic regression model using LASSO regularization and 5-fold cross validation for hyperparameter tuning. The model performed well on the test set, achieving a micro average area under the receiver operating characteristic and F-score of 0.94 (95% CI 0.93-0.95) and 0.77 (0.75-0.80) for GOS, and 0.90 (0.89-0.91) and 0.59 (0.57-0.62) for mRS, respectively. Our work demonstrates that an NLP algorithm can accurately assign neurologic outcomes based on free text clinical notes. This algorithm increases the scale of research on neurological outcomes that is possible with EHR data.
出院时的神经功能障碍水平是许多临床研究中的一项重要结果。在临床试验之外，神经功能结局通常必须通过对电子健康记录（EHR）中的临床记录进行劳动强度大的人工审查来提取。为了克服这一挑战，我们着手开发一种自然语言处理（NLP）方法，该方法可以自动读取临床记录以确定神经功能结局，从而使开展更大规模的神经功能结局研究成为可能。我们从2012年1月至2020年6月期间在波士顿两家大型医院住院的3632名患者中获取了7314份记录，包括出院小结（3485份）、职业治疗记录（1472份）和物理治疗记录（2357份）。14名临床专家对记录进行审查，以在格拉斯哥预后量表（GOS）上进行评分，该量表有4个类别，即“良好恢复”、“中度残疾”、“重度残疾”和“死亡”；并在改良Rankin量表（mRS）上进行评分，该量表有7个类别，即“无症状”、“无明显残疾”、“轻度残疾”、“中度残疾”、“中度重度残疾”、“重度残疾”和“死亡”。对于428名患者的记录，由2名专家对病例进行评分，得出GOS和mRS的评分者间信度估计值。在对记录进行预处理和特征提取后，我们使用LASSO正则化和5折交叉验证进行超参数调整，训练了一个多类逻辑回归模型。该模型在测试集上表现良好，GOS的受试者工作特征曲线下的微平均面积和F值分别为0.94（95%CI 0.93 - 0.95）和0.77（0.75 - 0.80），mRS的分别为0.90（0.89 - 0.91）和0.59（0.57 - 0.62）。我们的工作表明，一种NLP算法可以根据自由文本临床记录准确地确定神经功能结局。该算法扩大了利用EHR数据进行神经功能结局研究的规模。
