Infection Control Program, University of Geneva Hospitals and Faculty of Medicine, Geneva, Switzerland.
Departments of Infectious Diseases and Microbiology, Royal Prince Alfred Hospital, Sydney, Australia.
BMC Infect Dis. 2015 Feb 27;15:105. doi: 10.1186/s12879-015-0834-y.
Predictive models to identify unknown methicillin-resistant Staphylococcus aureus (MRSA) carriage on admission may optimise targeted MRSA screening and efficient use of resources. However, common approaches to model selection can result in overconfident estimates and poor predictive performance. We aimed to compare the performance of various models to predict previously unknown MRSA carriage on admission to surgical wards.
The study analysed data collected during a prospective cohort study which enrolled consecutive adult patients admitted to 13 surgical wards in 4 European hospitals. The participating hospitals were located in Athens (Greece), Barcelona (Spain), Cremona (Italy) and Paris (France). Universal admission MRSA screening was performed in the surgical wards. Data regarding demographic characteristics and potential risk factors for MRSA carriage were prospectively collected during the study period. Four logistic regression models were used to predict probabilities of unknown MRSA carriage using risk factor data: "Stepwise" (variables selected by backward elimination); "Best BMA" (model with highest posterior probability using Bayesian model averaging which accounts for uncertainty in model choice); "BMA" (average of all models selected with BMA); and "Simple" (model including variables selected >50% of the time by both Stepwise and BMA approaches applied to repeated random sub-samples of 50% of the data). To assess model performance, cross-validation against data not used for model fitting was conducted and net reclassification improvement (NRI) was calculated.
Of 2,901 patients enrolled, 111 (3.8%) were newly identified MRSA carriers. Recent hospitalisation and presence of a wound/ulcer were significantly associated with MRSA carriage in all models. While all models demonstrated limited predictive ability (mean c-statistics <0.7) the Simple model consistently detected more MRSA-positive individuals despite screening fewer patients than the Stepwise model. Moreover, the Simple model improved reclassification of patients into appropriate risk strata compared with the Stepwise model (NRI 6.6%, P = .07).
Though commonly used, models developed using stepwise variable selection can have relatively poor predictive value. When developing MRSA risk indices, simpler models, which account for uncertainty in model selection, may better stratify patients' risk of unknown MRSA carriage.
预测模型可用于识别入院时未知的耐甲氧西林金黄色葡萄球菌（MRSA）携带情况，从而优化针对 MRSA 的筛查，并提高资源利用效率。但是，常见的模型选择方法可能会导致过度自信的估计和较差的预测性能。我们旨在比较各种模型在预测外科病房新入院的未知 MRSA 携带情况方面的表现。
本研究分析了一项前瞻性队列研究的数据，该研究纳入了来自欧洲 4 家医院的 13 个外科病房的连续成年患者。参与医院分别位于希腊雅典、西班牙巴塞罗那、意大利克雷莫纳和法国巴黎。在外科病房进行了普遍的入院时 MRSA 筛查。研究期间，前瞻性地收集了有关人口统计学特征和 MRSA 携带潜在危险因素的数据。使用四个逻辑回归模型使用危险因素数据预测未知 MRSA 携带的概率：“逐步”（通过向后消除选择变量）；“最佳 BMA”（使用贝叶斯模型平均选择具有最高后验概率的模型，该模型考虑了模型选择的不确定性）；“BMA”（使用 BMA 选择的所有模型的平均值）；和“简单”（通过逐步和 BMA 方法应用于数据的 50%重复随机子样本中选择变量的时间超过 50%的模型）。为了评估模型性能，使用未用于模型拟合的数据进行了交叉验证，并计算了净重新分类改善（NRI）。
在纳入的 2901 名患者中，有 111 名（3.8%）新发现为 MRSA 携带者。最近的住院和存在伤口/溃疡与所有模型中的 MRSA 携带显著相关。虽然所有模型的预测能力均有限（平均 c 统计量<0.7），但与逐步模型相比，简单模型仍能检测到更多的 MRSA 阳性个体，尽管筛查的患者数量较少。此外，与逐步模型相比，简单模型在将患者重新分类到适当的风险分层方面有了改善（NRI 为 6.6%，P=0.07）。
虽然常用，但使用逐步变量选择方法开发的模型可能具有相对较差的预测价值。在开发 MRSA 风险指数时，更简单的模型可以更好地分层患者未知的 MRSA 携带风险，同时考虑到模型选择的不确定性。
