Syed Khajamoinuddin, Sleeman William, Hagan Michael, Palta Jatinder, Kapoor Rishabh, Ghosh Preetam
Department of Computer Science, Virginia Commonwealth University, Richmond, VA 23284, USA.
Department of Radiation Oncology, Virginia Commonwealth University, Richmond, VA 23298, USA.
Healthcare (Basel). 2020 Aug 14;8(3):272. doi: 10.3390/healthcare8030272.
The Radiotherapy Incident Reporting and Analysis System (RIRAS) receives incident reports from Radiation Oncology facilities across the US Veterans Health Affairs (VHA) enterprise and Virginia Commonwealth University (VCU). In this work, we propose a computational pipeline for analysis of radiation oncology incident reports. Our pipeline uses machine learning (ML) and natural language processing (NLP) based methods to predict the severity of the incidents reported in the RIRAS platform using the textual description of the reported incidents. These incidents in RIRAS are reviewed by a radiation oncology subject matter expert (SME), who initially triages some incidents based on the salient elements in the incident report. To automate the triage process, we used the data from the VHA treatment centers and the VCU radiation oncology department. We used NLP combined with traditional ML algorithms, including support vector machine (SVM) with linear kernel, and compared it against the transfer learning approach with the universal language model fine-tuning (ULMFiT) algorithm. In RIRAS, severities are divided into four categories; A, B, C, and D, with A being the most severe to D being the least. In this work, we built models to predict High (A & B) vs. Low (C & D) severity instead of all the four categories. Models were evaluated with macro-averaged precision, recall, and F1-Score. The Traditional ML machine learning (SVM-linear) approach did well on the VHA dataset with 0.78 F1-Score but performed poorly on the VCU dataset with 0.5 F1-Score. The transfer learning approach did well on both datasets with 0.81 F1-Score on VHA dataset and 0.68 F1-Score on the VCU dataset. Overall, our methods show promise in automating the triage and severity determination process from radiotherapy incident reports.
放射治疗事件报告与分析系统（RIRAS）接收来自美国退伍军人健康事务部（VHA）企业及弗吉尼亚联邦大学（VCU）各放射肿瘤学机构的事件报告。在本研究中，我们提出了一个用于分析放射肿瘤学事件报告的计算流程。我们的流程使用基于机器学习（ML）和自然语言处理（NLP）的方法，通过报告事件的文本描述来预测RIRAS平台上报事件的严重程度。RIRAS中的这些事件由放射肿瘤学主题专家（SME）进行审查，该专家最初会根据事件报告中的显著要素对一些事件进行分类。为了使分类过程自动化，我们使用了来自VHA治疗中心和VCU放射肿瘤学部门的数据。我们将NLP与传统ML算法相结合，包括使用线性核的支持向量机（SVM），并将其与采用通用语言模型微调（ULMFiT）算法的迁移学习方法进行比较。在RIRAS中，严重程度分为四类：A、B、C和D，A类最严重，D类最轻微。在本研究中，我们构建模型来预测高（A和B）与低（C和D）严重程度，而非所有四类。模型通过宏平均精度、召回率和F1分数进行评估。传统ML机器学习（SVM - 线性）方法在VHA数据集上表现良好，F1分数为0.78，但在VCU数据集上表现不佳，F1分数为0.5。迁移学习方法在两个数据集上均表现良好，在VHA数据集上F1分数为0.81，在VCU数据集上F1分数为0.68。总体而言，我们的方法在使放射治疗事件报告的分类和严重程度判定过程自动化方面显示出了前景。
