Manda S O, Gale C P, Hall A S, Gilthorpe M S
Biostatistics Unit, Medical Research Council, Pretoria, South Africa.
Cardiovasc J Afr. 2012 Nov;23(10):546-51. doi: 10.5830/CVJA-2011-064.
In order to improve the quality of care delivered to patients and to enable patient choice, public reports comparing hospital performances are routinely published. Robust systems of hospital 'report cards' on performance monitoring and evaluation are therefore crucial in medical decision-making processes. In particular, such systems should effectively account for and minimise systematic differences with regard to definitions and data quality, care and treatment quality, and 'case mix'.
Four methods for assessing hospital performance on mortality outcome measures were considered. The methods included combinations of Bayesian fixed- and random-effects models, and risk-adjusted mortality rate, and rank-based profiling techniques. The methods were empirically compared using 30-day mortality in patients admitted with acute coronary syndrome. Agreement was firstly assessed using median estimates between risk-adjusted mortality rates for a hospital and between ranks associated with a hospital's risk-adjusted mortality rates. Secondly, assessment of agreement was based on a classification of hospitals into low, normal or high performing using risk-adjusted mortality rates and ranks.
There was poor agreement between the point estimates of risk-adjusted mortality rates, but better agreement between ranks. However, for categorised performance, the observed agreement between the methods' classification of the hospital performance ranged from 90 to 98%. In only two of the six possible pair-wise comparisons was agreement reasonable, as reflected by a Kappa statistic; it was 0.71 between the methods of identifying outliers with the fixed-effect model and 0.77 with the hierarchical model. In the remaining four pair-wise comparisons, the agreement was, at best, moderate.
Even though the inconsistencies among the studied methods raise questions about which hospitals performed better or worse than others, it seems that the choice of the definition of outlying performance is less critical than that of the statistical approach. Therefore there is a need to find robust systems of 'regulation' or 'performance monitoring' that are meaningful to health service practitioners and providers.
为了提高提供给患者的护理质量并让患者能够做出选择，比较医院绩效的公开报告经常发布。因此，健全的医院“成绩单”系统，即绩效监测和评估系统，在医疗决策过程中至关重要。特别是，这样的系统应有效考虑并尽量减少在定义和数据质量、护理和治疗质量以及“病例组合”方面的系统差异。
考虑了四种评估医院死亡率结局指标绩效的方法。这些方法包括贝叶斯固定效应和随机效应模型的组合、风险调整死亡率以及基于排名的剖析技术。使用急性冠状动脉综合征患者的30天死亡率对这些方法进行实证比较。首先，使用医院风险调整死亡率的中位数估计值以及与医院风险调整死亡率相关的排名之间的中位数估计值来评估一致性。其次，基于使用风险调整死亡率和排名将医院分为低、中或高绩效类别来评估一致性。
风险调整死亡率的点估计值之间一致性较差，但排名之间的一致性较好。然而，对于分类绩效，各方法对医院绩效的分类之间观察到的一致性在90%至98%之间。在六种可能的两两比较中，只有两种比较的一致性是合理的，这由卡帕统计量反映；固定效应模型识别异常值的方法与分层模型之间的卡帕统计量为0.71，分层模型之间为0.77。在其余四种两两比较中，一致性充其量为中等。
尽管所研究方法之间的不一致性引发了关于哪些医院表现更好或更差的问题，但似乎异常绩效定义的选择不如统计方法的选择那么关键。因此，需要找到对卫生服务从业者和提供者有意义的健全的“监管”或“绩效监测”系统。
