Department of Biostatistics, Harvard T.H. Chan School of Public Health, Harvard University, Boston, Massachusetts.
Cardiovascular Division, Brigham and Women's Hospital, Harvard Medical School, Harvard University, Boston, Massachusetts.
JAMA Cardiol. 2023 Mar 1;8(3):290-295. doi: 10.1001/jamacardio.2022.5279.
For personalized or stratified medicine, it is critical to establish a reliable and efficient prediction model for a clinical outcome of interest. The goal is to develop a parsimonious model with fewer predictors for broad future application without compromising predictability. A general approach is to construct various empirical models via individual patients' specific baseline characteristics/biomarkers and then evaluate their relative merits. When the outcome of interest is the timing of a cardiovascular event, a commonly used metric to assess the adequacy of the fitted models is based on C statistics. These measures quantify a model's ability to separate those who develop events earlier from those who develop them later or not at all (discrimination), but they do not measure how closely model estimates match observed outcomes (prediction accuracy). Metrics that provide clinically interpretable measures to quantify prediction accuracy are needed.
C statistics measure the concordance between the risk scores derived from the model and the observed event time observations. However, C statistics do not quantify the model prediction accuracy. The integrated Brier Score, which calculates the mean squared distance between the empirical cumulative event-free curve and its individual patient's counterparts, estimates the prediction accuracy, but it is not clinically intuitive. A simple alternative measure is the average distance between the observed and predicted event times over the entire study population. This metric directly quantifies the model prediction accuracy and has often been used to evaluate the goodness of fit of the assumed models in settings other than survival data. This time-scale measure is easier to interpret than the C statistics or the Brier score.
This article enhances our understanding of the model selection/evaluation process with respect to prediction accuracy. A simple, intuitive measure for quantifying such accuracy beyond C statistics can improve the reliability and efficiency of the selected model for personalized and stratified medicine.
对于个性化或分层医学，建立可靠和高效的感兴趣临床结果预测模型至关重要。目标是开发一个具有较少预测因子的简约模型，以便在不影响可预测性的情况下广泛应用于未来。一种常用方法是通过个体患者的特定基线特征/生物标志物构建各种经验模型，然后评估它们的相对优势。当感兴趣的结果是心血管事件的时间时，评估拟合模型充分性的常用指标是基于 C 统计量。这些指标衡量模型区分早期发生事件和晚期或根本不发生事件的能力（区分能力），但它们不衡量模型估计与观察结果的接近程度（预测准确性）。需要提供具有临床可解释性的指标来量化预测准确性。
C 统计量衡量模型得出的风险评分与观察到的事件时间观测值之间的一致性。然而，C 统计量并不量化模型预测准确性。综合 Brier 评分计算经验累积无事件曲线与其个体患者对应曲线之间的均方距离，估计预测准确性，但它不具有临床直观性。一个简单的替代指标是整个研究人群中观察到的和预测的事件时间之间的平均距离。该指标直接量化模型预测准确性，并且经常用于评估在生存数据以外的设置中假设模型的拟合优度。该时间尺度指标比 C 统计量或 Brier 评分更容易解释。
本文增强了我们对预测准确性方面的模型选择/评估过程的理解。一种简单直观的方法可以量化超越 C 统计量的准确性，从而提高个性化和分层医学中所选模型的可靠性和效率。
