External validation of a predictive model for reintubation after cardiac surgery: A retrospective, observational study.

STUDY OBJECTIVE

Explore validation of a model to predict patients' risk of failing extubation, to help providers make informed, data-driven decisions regarding the optimal timing of extubation.

DESIGN

We performed temporal, geographic, and domain validations of a model for the risk of reintubation after cardiac surgery by assessing its performance on data sets from three academic medical centers, with temporal validation using data from the institution where the model was developed.

SETTING

Three academic medical centers in the United States.

PATIENTS

Adult patients arriving in the cardiac intensive care unit with an endotracheal tube in place after cardiac surgery.

INTERVENTIONS

Receiver operating characteristic (ROC) curves and concordance statistics were used as measures of discriminative ability, and calibration curves and Brier scores were used to assess the model's predictive ability.

MEASUREMENTS

Temporal validation was performed in 1642 patients with a reintubation rate of 4.8%, with the model demonstrating strong discrimination (optimism-corrected c-statistic 0.77) and low predictive error (Brier score 0.044) but poor model precision and recall (Optimal F1 score 0.29). Combined domain and geographic validation were performed in 2041 patients with a reintubation rate of 1.5%. The model displayed solid discriminative ability (optimism-corrected c-statistic = 0.73) and low predictive error (Brier score = 0.0149) but low precision and recall (Optimal F1 score = 0.13). Geographic validation was performed in 2489 patients with a reintubation rate of 1.6%, with the model displaying good discrimination (optimism-corrected c-statistic = 0.71) and predictive error (Brier score = 0.0152) but poor precision and recall (Optimal F1 score = 0.13).

MAIN RESULTS

The reintubation model displayed strong discriminative ability and low predictive error within each validation cohort.

CONCLUSIONS

Future work is needed to explore how to optimize models before local implementation.