Ludwig-Maximilians-Universität München, Institut für Medizinische Informationsverarbeitung, Biometrie und Epidemiologie, München, Germany.
BMC Health Serv Res. 2011 May 18;11:106. doi: 10.1186/1472-6963-11-106.
Hospitals are increasingly being evaluated with respect to the quality of provided care. In this setting, several indicator sets compete with one another for the assessment of effectiveness and safety. However, there have been few comparative investigations covering different sets. The objective of this study was to answer three questions: How concordant are different indicator sets on a hospital level? What is the effect of applying different reference values? How stable are the positions of a hospital ranking?
Routine data were made available to three companies offering the Patient Safety Indicators, an indicator set from the HELIOS Hospital Group, and measurements based on Disease Staging™. Ten hospitals from North Rhine-Westphalia, comprising a total of 151,960 inpatients in 2006, volunteered to participate in this study. The companies provided standard quality reports for the ten hospitals. Composite measures were defined for strengths and weaknesses. In addition to the different indicator sets, different reference values for one set allowed the construction of several comparison groups. Concordance and robustness were analyzed using the non-parametric correlation coefficient and Kendall's W.
Indicator sets differing only in the reference values of the indicators showed significant correlations in most of the pairs with respect to weaknesses (maximum r = 0.927, CI 0.714-0.983, p < 0.001). There were also significant correlations between different sets (maximum r = 0.829, CI 0.417-0.958, p = 0.003) having different indicators or when different methods for performance assessment were applied. The results were weaker measuring hospital strengths (maximum r = 0.669, CI 0.068-0.914, p = 0.034). In a hospital ranking, only two hospitals belonged consistently either to the superior or to the inferior half of the group. Even altering reference values or the supplier for the same indicator set changed the rank for nine out of ten hospitals.
Our results reveal an unsettling lack of concordance in estimates of hospital performance when different quality indicator sets are used. These findings underline the lack of consensus regarding optimal validated measures for judging hospital quality. The indicator sets shared a common definition of quality, independent of their focus on patient safety, mortality, or length of stay. However, for most of the hospitals, changing the indicator set or the reference value resulted in a shift from the superior to the inferior half of the group or vice versa. Thus, while taken together the indicator sets offer the hospitals complementary pictures of their quality, on an individual basis they do not establish a reliable ranking.
医院的医疗质量正越来越受到重视。在这种情况下，有几个指标集在评估有效性和安全性方面相互竞争。然而，对于不同的指标集，很少有比较研究。本研究的目的是回答三个问题：不同的指标集在医院层面上的一致性如何？应用不同的参考值有什么影响？医院排名的稳定性如何？
为三家提供患者安全指标的公司、一家 HELIOS 医院集团的指标集以及基于疾病分期的测量方法提供了常规数据。北莱茵-威斯特法伦州的 10 家医院自愿参与了这项研究，共有 151960 名住院患者。这些公司为这 10 家医院提供了标准的质量报告。为优势和劣势定义了综合指标。除了不同的指标集外，一个指标集的不同参考值允许构建几个比较组。使用非参数相关系数和 Kendall 的 W 来分析一致性和稳健性。
在考虑到指标的参考值不同的情况下，具有不同指标或采用不同绩效评估方法的不同指标集之间也存在显著相关性（最大 r = 0.829，CI 0.417-0.958，p = 0.003）。对于劣势（最大 r = 0.927，CI 0.714-0.983，p < 0.001），大多数情况下，只有参考值不同的指标集显示出显著相关性。对于医院优势（最大 r = 0.669，CI 0.068-0.914，p = 0.034），相关性较弱。在医院排名中，只有两家医院始终属于上半区或下半区。即使改变参考值或同一指标集的供应商，也会导致十分之九的医院的排名发生变化。
我们的结果显示，当使用不同的质量指标集时，医院绩效评估的估计存在令人不安的不一致性。这些发现强调了对于判断医院质量的最佳验证指标缺乏共识。这些指标集具有共同的质量定义，与它们关注患者安全、死亡率或住院时间无关。然而，对于大多数医院来说，改变指标集或参考值会导致它们从上半区转移到下半区，或者反之亦然。因此，虽然这些指标集共同为医院提供了其质量的互补画面，但在个体基础上，它们并不能建立可靠的排名。
