Zanotto Bruna Stella, Beck da Silva Etges Ana Paula, Dal Bosco Avner, Cortes Eduardo Gabriel, Ruschel Renata, De Souza Ana Claudia, Andrade Claudio M V, Viegas Felipe, Canuto Sergio, Luiz Washington, Ouriques Martins Sheila, Vieira Renata, Polanczyk Carisi, André Gonçalves Marcos
National Institute of Health Technology Assessment - INCT/IATS (CNPQ 465518/2014-1), Universidade Federal do Rio Grande do Sul, Porto Alegre, Brazil.
Graduate Program in Epidemiology, Universidade Federal do Rio Grande do Sul, Porto Alegre, Brazil.
JMIR Med Inform. 2021 Nov 1;9(11):e29120. doi: 10.2196/29120.
With the rapid adoption of electronic medical records (EMRs), there is an ever-increasing opportunity to collect data and extract knowledge from EMRs to support patient-centered stroke management.
This study aims to compare the effectiveness of state-of-the-art automatic text classification methods in classifying data to support the prediction of clinical patient outcomes and the extraction of patient characteristics from EMRs.
Our study addressed the computational problems of information extraction and automatic text classification. We identified essential tasks to be considered in an ischemic stroke value-based program. The 30 selected tasks were classified (manually labeled by specialists) according to the following value agenda: tier 1 (achieved health care status), tier 2 (recovery process), care related (clinical management and risk scores), and baseline characteristics. The analyzed data set was retrospectively extracted from the EMRs of patients with stroke from a private Brazilian hospital between 2018 and 2019. A total of 44,206 sentences from free-text medical records in Portuguese were used to train and develop 10 supervised computational machine learning methods, including state-of-the-art neural and nonneural methods, along with ontological rules. As an experimental protocol, we used a 5-fold cross-validation procedure repeated 6 times, along with subject-wise sampling. A heatmap was used to display comparative result analyses according to the best algorithmic effectiveness (F1 score), supported by statistical significance tests. A feature importance analysis was conducted to provide insights into the results.
The top-performing models were support vector machines trained with lexical and semantic textual features, showing the importance of dealing with noise in EMR textual representations. The support vector machine models produced statistically superior results in 71% (17/24) of tasks, with an F1 score >80% regarding care-related tasks (patient treatment location, fall risk, thrombolytic therapy, and pressure ulcer risk), the process of recovery (ability to feed orally or ambulate and communicate), health care status achieved (mortality), and baseline characteristics (diabetes, obesity, dyslipidemia, and smoking status). Neural methods were largely outperformed by more traditional nonneural methods, given the characteristics of the data set. Ontological rules were also effective in tasks such as baseline characteristics (alcoholism, atrial fibrillation, and coronary artery disease) and the Rankin scale. The complementarity in effectiveness among models suggests that a combination of models could enhance the results and cover more tasks in the future.
Advances in information technology capacity are essential for scalability and agility in measuring health status outcomes. This study allowed us to measure effectiveness and identify opportunities for automating the classification of outcomes of specific tasks related to clinical conditions of stroke victims, and thus ultimately assess the possibility of proactively using these machine learning techniques in real-world situations.
随着电子病历（EMR）的迅速普及，从电子病历中收集数据并提取知识以支持以患者为中心的中风管理的机会越来越多。
本研究旨在比较最先进的自动文本分类方法在对数据进行分类以支持临床患者结局预测和从电子病历中提取患者特征方面的有效性。
我们的研究解决了信息提取和自动文本分类的计算问题。我们确定了基于缺血性中风价值的计划中要考虑的基本任务。根据以下价值议程对选定的30项任务进行分类（由专家手动标注）：一级（实现的医疗保健状态）、二级（恢复过程）、护理相关（临床管理和风险评分）以及基线特征。分析的数据集是从2018年至2019年巴西一家私立医院中风患者的电子病历中回顾性提取的。总共44206条来自葡萄牙语自由文本病历的句子用于训练和开发10种有监督的计算机器学习方法，包括最先进的神经和非神经方法以及本体规则。作为实验方案，我们使用了重复6次的5折交叉验证程序以及按受试者抽样。使用热图根据最佳算法有效性（F1分数）显示比较结果分析，并辅以统计显著性检验。进行了特征重要性分析以深入了解结果。
表现最佳的模型是使用词汇和语义文本特征训练的支持向量机，这表明处理电子病历文本表示中的噪声的重要性。支持向量机模型在71%（17/24）的任务中产生了统计学上更优的结果，在护理相关任务（患者治疗地点、跌倒风险、溶栓治疗和压疮风险）、恢复过程（口服进食或行走及沟通能力）、实现的医疗保健状态（死亡率）以及基线特征（糖尿病、肥胖、血脂异常和吸烟状况）方面F1分数>80%。鉴于数据集的特征，神经方法在很大程度上被更传统的非神经方法超越。本体规则在基线特征（酗酒、心房颤动和冠状动脉疾病）和Rankin量表等任务中也有效。模型之间有效性的互补性表明，模型组合可能会在未来提高结果并涵盖更多任务。
信息技术能力的进步对于衡量健康状况结局的可扩展性和敏捷性至关重要。本研究使我们能够衡量有效性，并确定自动分类与中风患者临床状况相关的特定任务结局的机会，从而最终评估在实际情况中主动使用这些机器学习技术的可能性。
