Subgroup mortality probability models: are they necessary for specialized intensive care units?

OBJECTIVES

To examine the sensitivity of the performance of the latest Mortality Probability Model at intensive care unit admission (MPM0-III) to case-mix variations and to determine how specialized models for these subgroups would affect intensive care unit performance assessment. MPM0-III is an important benchmarking tool for intensive care units in Project IMPACT. Overall, MPM0-III has excellent discrimination and calibration but its performance varies on six common patient subsets.

DESIGN

A total of 124,171 patients in six subgroups (complex cardiovascular, trauma, elective surgery, medical, neurosurgery, and emergency surgery) were divided randomly into development (60%) and validation (40%) groups. A logistic regression model was developed to predict hospital mortality for each subgroup, using MPM0-III variables. Model performance was evaluated on the validation sets, using Hosmer-Lemeshow and receiver operating characteristic statistics. Intensive care unit standardized mortality ratios, using the subgroup models and MPM0-III, were compared. A sensitivity analysis was used to identify the occurrence of each subgroup associated with degraded MPM0-III performance.

SETTING

One hundred thirty-five intensive care units at 98 hospitals participating in Project IMPACT between 2001 and 2004. ICUs with <100 patient records were excluded.

PATIENTS

Consecutive intensive care unit patients in the Project IMPACT database who were eligible for MPM0-II scoring.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

Hospital mortality and standardized mortality ratio values by intensive care unit. All six subgroup models had good performance on their validation sets. Intensive care unit standardized mortality ratios calculated with MPM0-III and the subgroup models were nearly identical, with MPM0-III identifying 33 of 135 as significant standardized mortality ratio outliers and the subgroup models identifying 35 of 135, with 33 overlapping. Sensitivity analysis indicated that MPM0-III calibration degraded substantially only when patient mix varied significantly from that of the data set on which MPM0-III was based.

CONCLUSION

We recommend users of MPM make MPM0-III their primary model. Subgroup models may have utility when evaluating highly specialized intensive care units or in research on specific, homogeneous populations.