Jimenez-Maggiora Gustavo A, Donohue Michael C, Rafii Michael S, Raman Rema, Aisen Paul S
Alzheimer's Therapeutic Research Institute, University of Southern California, San Diego, CA, United States.
Alzheimer's Therapeutic Research Institute, University of Southern California, San Diego, CA, United States.
J Prev Alzheimers Dis. 2025 Jan;12(1):100002. doi: 10.1016/j.tjpad.2024.100002. Epub 2025 Jan 1.
Investigators conducting clinical trials have an ethical, scientific, and regulatory obligation to protect the safety of trial participants. Traditionally, safety monitoring includes manual review and coding of adverse event data by expert clinicians.
Our study explores the use of natural language processing (NLP) and artificial intelligence (AI) methods to streamline and standardize clinician coding of adverse event data in Alzheimer's disease (AD) clinical trials.
Our quantitative retrospective study aimed to develop a gold standard AD adverse event data set, evaluate the predictive performance of NLP-based models to classify adverse events, and determine whether automated coding is more efficient, accurate, reliable, and consistent than clinician coding.
Our study was conducted at the University of Southern California's Alzheimer's Therapeutic Research Institute (ATRI). ATRI serves as the clinical and data coordinating center for the Alzheimer's Clinical Trial Consortium (ACTC).
We collected demographic and adverse event data from eight completed clinical trials in participants (n=1920) with symptomatic AD conducted between 2005 and 2020.
Original expert clinician-confirmed codes were used for all model performance comparisons. F1 score was used as the primary model selection metric. Final classifier performance was evaluated using predictive accuracy. Clinician effort was measured in time to code, review, and confirm coded adverse events.
In a sample of 1000 adverse events, AI-based AE coding achieved higher accuracy (∼20% increase in accuracy) and was more cost-effective (∼80% cost reduction) than traditional clinician coding.
Our study results demonstrate how approaches that effectively combine AI and human expertise can improve the efficiency and quality of adverse event coding and clinical trial safety monitoring.
开展临床试验的研究人员在伦理、科学和监管方面有保护试验参与者安全的义务。传统上，安全监测包括由专家临床医生对手动审查和不良事件数据进行编码。
我们的研究探索使用自然语言处理（NLP）和人工智能（AI）方法来简化和规范阿尔茨海默病（AD）临床试验中临床医生对不良事件数据的编码。
我们的定量回顾性研究旨在开发一个AD不良事件数据集金标准，评估基于NLP的模型对不良事件进行分类的预测性能，并确定自动编码是否比临床医生编码更高效、准确、可靠和一致。
我们的研究在南加州大学阿尔茨海默病治疗研究所（ATRI）进行。ATRI是阿尔茨海默病临床试验联盟（ACTC）的临床和数据协调中心。
我们从2005年至2020年期间对1920名有症状AD参与者进行的八项已完成临床试验中收集了人口统计学和不良事件数据。
所有模型性能比较均使用原始专家临床医生确认的编码。F1分数用作主要的模型选择指标。使用预测准确性评估最终分类器的性能。通过编码、审查和确认编码不良事件的时间来衡量临床医生的工作量。
在1000例不良事件样本中，基于AI的不良事件编码比传统临床医生编码具有更高的准确性（准确性提高约20%）且更具成本效益（成本降低约80%）。
我们的研究结果表明，有效结合AI和人类专业知识的方法如何能够提高不良事件编码和临床试验安全监测的效率和质量。
