Collin Lindsay J, Riis Anders H, MacLehose Richard F, Ahern Thomas P, Erichsen Rune, Thorlacius-Ussing Ole, Lash Timothy L
Department of Epidemiology, Emory University, Atlanta, GA, USA.
Department of Clinical Epidemiology, Aarhus University Hospital, Aarhus, Denmark.
Clin Epidemiol. 2020 Feb 3;12:113-121. doi: 10.2147/CLEP.S230314. eCollection 2020.
Among men and women diagnosed with colorectal cancer (CRC), 20-50% will develop a cancer recurrence. Cancer recurrences are not routinely captured by most population-based registries; however, linkage across Danish registries allows for the development of predictive models to detect recurrence. Successful application of such models in population-based settings requires validation against a gold standard to ensure the accuracy of recurrence identification.
We apply a recently developed validation study design for prospectively collected validation data to validate predicted CRC recurrences against gold standard diagnoses from medical records in an actively followed cohort of CRC patients in Denmark.
We use a Bayesian monitoring framework, traditionally used in clinical trials, to iteratively update classification parameters (positive and negative predictive values, and sensitivity and specificity) in an adaptive validation substudy design. This design allows determination of the sample size necessary to estimate the corresponding parameters and to identify when validation efforts can cease based on predefined criteria for parameter values and levels of precision.
Among 355 men and women diagnosed with CRC in Denmark and actively followed semi-annually, there were 63 recurrences diagnosed by active follow-up and 70 recurrences identified by a predictive algorithm. The adaptive validation design met stopping criteria for the classification parameters after 120 patients had their recurrence information validated. This stopping point yielded parameter estimates for the classification parameters similar to those obtained when the entire cohort was validated, with 66% less patients needed for the validation study.
In this proof of concept application of the adaptive validation study design for outcome misclassification, we demonstrated the ability of the method to accurately determine when sufficient validation data have been collected. This method serves as a novel validation substudy design for prospectively collected data with simultaneous implementation of a validation study.
在被诊断为结直肠癌（CRC）的男性和女性中，20%-50%会出现癌症复发。大多数基于人群的登记系统通常不会记录癌症复发情况；然而，丹麦各登记系统之间的关联使得能够开发预测模型来检测复发。要在基于人群的环境中成功应用此类模型，需要对照金标准进行验证，以确保复发识别的准确性。
我们应用一种最近开发的验证研究设计，用于前瞻性收集验证数据，以对照丹麦一组积极随访的CRC患者病历中的金标准诊断，验证预测的CRC复发情况。
我们使用一种传统上用于临床试验的贝叶斯监测框架，在自适应验证子研究设计中迭代更新分类参数（阳性和阴性预测值以及敏感性和特异性）。这种设计能够确定估计相应参数所需的样本量，并根据参数值和精度水平的预定义标准确定何时可以停止验证工作。
在丹麦355名被诊断为CRC且每半年进行一次积极随访的男性和女性中，通过积极随访诊断出63例复发，通过预测算法识别出70例复发。在120名患者的复发信息得到验证后，自适应验证设计达到了分类参数的停止标准。这个停止点产生的分类参数估计值与对整个队列进行验证时获得的估计值相似，而验证研究所需的患者数量减少了66%。
在这种针对结果错误分类的自适应验证研究设计的概念验证应用中，我们证明了该方法能够准确确定何时已收集到足够的验证数据。这种方法是一种新颖的验证子研究设计，用于前瞻性收集数据并同时开展验证研究。
