Department of Radiology and Nuclear Medicine, Erasmus MC, University Medical Center Rotterdam, P.O. Box 2040, 3000 CA, Rotterdam, The Netherlands.
Department of Medical Informatics, Erasmus MC, University Medical Center Rotterdam, P.O. Box 2040, 3000 CA, Rotterdam, The Netherlands.
Eur Radiol. 2022 Jun;32(6):3996-4002. doi: 10.1007/s00330-021-08467-8. Epub 2022 Jan 6.
To develop and validate classifiers for automatic detection of actionable findings and documentation of nonroutine communication in routinely delivered radiology reports.
Two radiologists annotated all actionable findings and communication mentions in a training set of 1,306 radiology reports and a test set of 1,000 reports randomly selected from the electronic health record system of a large tertiary hospital. Various feature sets were constructed based on the impression section of the reports using different preprocessing steps (stemming, removal of stop words, negations, and previously known or stable findings) and n-grams. Random forest classifiers were trained to detect actionable findings, and a decision-rule classifier was trained to find communication mentions. Classifier performance was evaluated by the area under the receiver operating characteristic curve (AUC), sensitivity, and specificity.
On the training set, the actionable finding classifier with the highest cross-validated performance was obtained for a feature set of unigrams, after stemming and removal of negated, known, and stable findings. On the test set, this classifier achieved an AUC of 0.876 (95% CI 0.854-0.898). The classifier for communication detection was trained after negation removal, using unigrams as features. The resultant decision rule had a sensitivity of 0.841 (95% CI 0.706-0.921) and specificity of 0.990 (95% CI 0.981-0.994) on the test set.
Automatic detection of actionable findings and subsequent communication in routinely delivered radiology reports is possible. This can serve quality control purposes and may alert radiologists to the presence of actionable findings during reporting.
• Classifiers were developed for automatic detection of the broad spectrum of actionable findings and subsequent communication mentions in routinely delivered radiology reports. • Straightforward report preprocessing and simple feature sets can produce well-performing classifiers. • The resultant classifiers show good performance for detection of actionable findings and excellent performance for detection of communication mentions.
开发并验证用于自动检测可操作发现并记录常规提供的放射学报告中非例行沟通的分类器。
两名放射科医生对来自大型三级医院电子健康记录系统的 1306 份放射学报告的训练集和 1000 份随机选择的测试集的所有可操作发现和沟通提及进行注释。基于报告的印象部分，使用不同的预处理步骤（词干提取、停用词去除、否定词去除以及先前已知或稳定的发现）和 n-gram 构建各种特征集。使用随机森林分类器来检测可操作的发现，并训练决策规则分类器来寻找沟通提及。通过接收者操作特征曲线（AUC）、灵敏度和特异性来评估分类器的性能。
在训练集上，性能最高的可操作发现分类器是基于一元词特征集，在进行词干提取和去除否定词、已知和稳定发现后获得的。在测试集上，该分类器的 AUC 为 0.876（95%置信区间 0.854-0.898）。沟通检测分类器是在去除否定词后，使用一元词作为特征训练的。在测试集上，该决策规则的灵敏度为 0.841（95%置信区间 0.706-0.921），特异性为 0.990（95%置信区间 0.981-0.994）。
自动检测常规提供的放射学报告中的可操作发现和随后的沟通是可能的。这可以用于质量控制目的，并可能在报告过程中提醒放射科医生注意可操作发现的存在。
开发了用于自动检测常规提供的放射学报告中广泛的可操作发现和随后的沟通提及的分类器。
简单的报告预处理和特征集可以产生性能良好的分类器。
生成的分类器在检测可操作发现方面表现良好，在检测沟通提及方面表现出色。
