Artificial intelligence-enabled safety monitoring in Alzheimer's disease clinical trials.

BACKGROUND

Investigators conducting clinical trials have an ethical, scientific, and regulatory obligation to protect the safety of trial participants. Traditionally, safety monitoring includes manual review and coding of adverse event data by expert clinicians.

OBJECTIVES

Our study explores the use of natural language processing (NLP) and artificial intelligence (AI) methods to streamline and standardize clinician coding of adverse event data in Alzheimer's disease (AD) clinical trials.

DESIGN

Our quantitative retrospective study aimed to develop a gold standard AD adverse event data set, evaluate the predictive performance of NLP-based models to classify adverse events, and determine whether automated coding is more efficient, accurate, reliable, and consistent than clinician coding.

SETTING

Our study was conducted at the University of Southern California's Alzheimer's Therapeutic Research Institute (ATRI). ATRI serves as the clinical and data coordinating center for the Alzheimer's Clinical Trial Consortium (ACTC).

PARTICIPANTS

We collected demographic and adverse event data from eight completed clinical trials in participants (n=1920) with symptomatic AD conducted between 2005 and 2020.

MEASUREMENTS

Original expert clinician-confirmed codes were used for all model performance comparisons. F1 score was used as the primary model selection metric. Final classifier performance was evaluated using predictive accuracy. Clinician effort was measured in time to code, review, and confirm coded adverse events.

RESULTS

In a sample of 1000 adverse events, AI-based AE coding achieved higher accuracy (∼20% increase in accuracy) and was more cost-effective (∼80% cost reduction) than traditional clinician coding.

CONCLUSIONS

Our study results demonstrate how approaches that effectively combine AI and human expertise can improve the efficiency and quality of adverse event coding and clinical trial safety monitoring.