National Institute of Health Data Science, Peking University, 38 Xueyuan Road, Beijing, 100191, China.
Department of Mathematics, University of British Columbia, Vancouver, V6T1Z, BC, Canada.
Sci Rep. 2022 Apr 26;12(1):6777. doi: 10.1038/s41598-022-10887-5.
A large number of adverse drug reaction (ADR) reports are collected yearly through the spontaneous report system (SRS). However, experienced experts from ADR monitoring centers (ADR experts, hereafter) reviewed only a few reports based on current policies. Moreover, the causality assessment of ADR reports was conducted according to the official approach based on the WHO-UMC system, a knowledge- and labor-intensive task that highly relies on an individual's expertise. Our objective is to devise a method to automatically assess ADR reports and support the efficient exploration of ADRs interactively. Our method could improve the capability to assess and explore a large volume of ADR reports and aid reporters in self-improvement. We proposed a workflow for assisting the assessment of ADR reports by combining an automatic assessment prediction model and a human-centered interactive visualization method. Our automatic causality assessment model (ACA model)-an ordinal logistic regression model-automatically assesses ADR reports under the current causality category. Based on the results of the ACA model, we designed a warning signal to indicate the degree of the anomaly of ADR reports. An interactive visualization technique was used for exploring and examining reports extended by automatic assessment of the ACA model and the warning signal. We applied our method to the SRS report dataset of the year 2019, collected in Guangdong province, China. Our method is evaluated by comparing automatic assessments by the ACA model to ADR reports labeled by ADR experts, i.e., the ground truth results from the multinomial logistic regression and the decision tree. The ACA model achieves an accuracy of 85.99%, a multiclass macro-averaged area under the curve (AUC) of 0.9572, while the multinomial logistics regression and decision tree yield 80.82%, 0.8603, and 85.39%, 0.9440, respectively, on the testing set. The new warning signal is able to assist ADR experts to quickly focus on reports of interest with our interactive visualzation tool. Reports of interest that are selected with high scores of the warning signal are analyzed in details by an ADR expert. The usefulness of the overall method is further evaluated through the interactive analysis of the data by ADR expert. Our ACA model achieves good performance and is superior to the multinomial logistics and the decision tree. The warning signal we designed allows efficient filtering of the full ADR reports down to much fewer reports showing anomalies. The usefulness of our interactive visualization is demonstrated by examples of unusual reports that are quickly identified. Our overall method could potentially improve the capability of analyzing ADR reports and reduce human labor and the chance of missing critical reports.
大量的药物不良反应（ADR）报告通过自发报告系统（SRS）每年收集。然而，根据现行政策，ADR 监测中心的经验丰富的专家仅审查了少数报告。此外，ADR 报告的因果关系评估是根据基于世界卫生组织-药物不良反应监测中心（WHO-UMC）系统的官方方法进行的，这是一项知识和劳动密集型任务，高度依赖个人的专业知识。我们的目标是设计一种自动评估 ADR 报告并支持交互式高效探索 ADR 的方法。我们的方法可以提高评估和探索大量 ADR 报告的能力，并帮助报告人自我提高。我们提出了一种通过结合自动评估预测模型和以人为中心的交互可视化方法来辅助 ADR 报告评估的工作流程。我们的自动因果关系评估模型（ACA 模型）-一个有序逻辑回归模型-根据当前的因果关系类别自动评估 ADR 报告。基于 ACA 模型的结果，我们设计了一个警告信号来指示 ADR 报告异常的程度。交互可视化技术用于探索和检查由 ACA 模型自动评估和警告信号扩展的报告。我们将我们的方法应用于 2019 年在中国广东省收集的 SRS 报告数据集。我们的方法通过将 ACA 模型的自动评估与 ADR 专家标记的 ADR 报告进行比较来进行评估，即来自多项逻辑回归和决策树的地面真实结果。ACA 模型在测试集上的准确率为 85.99%，多类宏平均曲线下面积（AUC）为 0.9572，而多项逻辑回归和决策树的准确率分别为 80.82%、0.8603 和 85.39%、0.9440。新的警告信号能够帮助 ADR 专家使用我们的交互可视化工具快速关注感兴趣的报告。选择高警告信号得分的感兴趣报告由 ADR 专家进行详细分析。ADR 专家通过对数据的交互分析进一步评估了整体方法的有效性。我们的 ACA 模型表现良好，优于多项逻辑回归和决策树。我们设计的警告信号允许有效地过滤掉所有 ADR 报告，只留下显示异常的报告少得多。通过快速识别异常报告的示例，展示了我们交互可视化的有用性。我们的整体方法有可能提高分析 ADR 报告的能力，并减少人力和遗漏关键报告的机会。
