Al-Farra Hatem, de Mol Bas A J M, Ravelli Anita C J, Ter Burg W J P P, Houterman Saskia, Henriques José P S, Abu-Hanna Ameen, Vis M M, Vos J, Timmers L, Tonino W A L, Schotborgh C E, Roolvink V, Porta F, Stoel M G, Kats S, Amoroso G, van der Werf H W, Stella P R, de Jaegere P
Department of Medical Informatics, Amsterdam UMC, University of Amsterdam, Amsterdam, the Netherlands.
Heart Center, Amsterdam UMC, University of Amsterdam, Amsterdam, the Netherlands.
Int J Cardiol Heart Vasc. 2021 Jan 23;32:100716. doi: 10.1016/j.ijcha.2021.100716. eCollection 2021 Feb.
The predictive performance of the models FRANCE-2 and ACC-TAVI for early-mortality after Transcatheter Aortic Valve Implantation (TAVI) can decline over time and can be enhanced by updating them on new populations. We aim to update and internally and temporally validate these models using a recent TAVI-cohort from the Netherlands Heart Registration (NHR).
We used data of TAVI-patients treated in 2013-2017. For each original-model, the best update-method (model-intercept, model-recalibration, or model-revision) was selected by a closed-testing procedure. We internally validated both updated models with 1000 bootstrap samples. We also updated the models on the 2013-2016 dataset and temporally validated them on the 2017-dataset. Performance measures were the Area-Under ROC-curve (AU-ROC), Brier-score, and calibration graphs.
We included 6177 TAVI-patients, with 4.5% observed early-mortality. The selected update-method for FRANCE-2 was model-intercept-update. Internal validation showed an AU-ROC of 0.63 (95%CI 0.62-0.66) and Brier-score of 0.04 (0.04-0.05). Calibration graphs show that it overestimates early-mortality. In temporal-validation, the AU-ROC was 0.61 (0.53-0.67).The selected update-method for ACC-TAVI was model-revision. In internal-validation, the AU-ROC was 0.63 (0.63-0.66) and Brier-score was 0.04 (0.04-0.05). The updated ACC-TAVI calibrates well up to a probability of 20%, and subsequently underestimates early-mortality. In temporal-validation the AU-ROC was 0.65 (0.58-0.72).
Internal-validation of the updated models FRANCE-2 and ACC-TAVI with data from the NHR demonstrated improved performance, which was better than in external-validation studies and comparable to the original studies. In temporal-validation, ACC-TAVI outperformed FRANCE-2 because it suffered less from changes over time.
经导管主动脉瓣植入术（TAVI）后，FRANCE-2和ACC-TAVI模型对早期死亡率的预测性能可能会随时间下降，通过在新人群中对其进行更新可提高预测性能。我们旨在使用荷兰心脏注册（NHR）中最近的TAVI队列对这些模型进行更新，并在内部和时间上进行验证。
我们使用了2013年至2017年接受TAVI治疗患者的数据。对于每个原始模型，通过封闭测试程序选择最佳更新方法（模型截距、模型重新校准或模型修订）。我们使用1000个自抽样样本对两个更新后的模型进行内部验证。我们还在2013 - 2016数据集上更新模型，并在2017数据集上进行时间验证。性能指标包括ROC曲线下面积（AU-ROC）、Brier评分和校准图。
我们纳入了6177例TAVI患者，观察到的早期死亡率为4.5%。FRANCE-2模型选择的更新方法是模型截距更新。内部验证显示AU-ROC为0.63（95%CI 0.62 - 0.66），Brier评分为0.04（0.04 - 0.05）。校准图显示它高估了早期死亡率。在时间验证中，AU-ROC为0.61（0.53 - 0.67）。ACC-TAVI模型选择的更新方法是模型修订。在内部验证中，AU-ROC为0.63（0.63 - 0.66），Brier评分为0.04（0.04 - 0.05）。更新后的ACC-TAVI在概率达到20%之前校准良好，随后低估了早期死亡率。在时间验证中，AU-ROC为0.65（0.58 - 0.72）。
使用NHR数据对更新后的FRANCE-2和ACC-TAVI模型进行内部验证显示性能有所改善，优于外部验证研究，且与原始研究相当。在时间验证中，ACC-TAVI的表现优于FRANCE-2，因为它受时间变化的影响较小。
