Qi Shi-Ang, Kumar Neeraj, Farrokh Mahtab, Sun Weijie, Kuan Li-Hao, Ranganath Rajesh, Henao Ricardo, Greiner Russell
Computing Science, University of Alberta, Edmonton, Canada.
Alberta Machine Intelligence Institute, Edmonton, Canada.
Proc Mach Learn Res. 2023 Jul;202:28244-28276.
One straightforward metric to evaluate a survival prediction model is based on the Mean Absolute Error (MAE) - the average of the absolute difference between the time predicted by the model and the true event time, over all subjects. Unfortunately, this is challenging because, in practice, the test set includes (right) censored individuals, meaning we do not know when a censored individual actually experienced the event. In this paper, we explore various metrics to estimate MAE for survival datasets that include (many) censored individuals. Moreover, we introduce a novel and effective approach for generating realistic semi-synthetic survival datasets to facilitate the evaluation of metrics. Our findings, based on the analysis of the semi-synthetic datasets, reveal that our proposed metric (MAE using pseudo-observations) is able to rank models accurately based on their performance, and often closely matches the true MAE - in particular, is better than several alternative methods.
评估生存预测模型的一个直接指标是基于平均绝对误差（MAE）——模型预测时间与真实事件时间之间绝对差值的平均值，涵盖所有受试者。不幸的是，这具有挑战性，因为在实际中，测试集包含（右）删失个体，这意味着我们不知道删失个体实际何时经历该事件。在本文中，我们探索了各种指标来估计包含（众多）删失个体的生存数据集的MAE。此外，我们引入了一种新颖且有效的方法来生成逼真的半合成生存数据集，以促进指标评估。基于对半合成数据集的分析，我们的研究结果表明，我们提出的指标（使用伪观测值的MAE）能够根据模型性能准确地对模型进行排名，并且通常与真实MAE非常接近——特别是，优于几种替代方法。
