Scinto J D, Galusha D H, Krumholz H M, Meehan T P
Qualidigm, Middletown, CT 06457, USA.
J Clin Epidemiol. 2001 Nov;54(11):1103-11. doi: 10.1016/s0895-4356(01)00381-x.
To demonstrate the importance of evaluating overall quality indicator reliability, in addition to component or variable level reliability, a comparison of interrater agreement on four chart-abstracted pneumonia-related processes of care was conducted. The hospital medical records of 356 Medicare patients' recent discharges for pneumonia were independently abstracted by different abstractors. Kappa, prevalence and bias-adjusted kappa, P(pos), P(neg), and the Bias Index were used to assess reliability of composite quality indicators and their components. The adjusted kappas for the data elements used to determine eligibility to receive as well as to derive the pneumonia-related processes of care ranged from 0.68 to 1.0. The adjusted kappa associated with overall eligibility to receive the pneumonia-related processes of care was 0.63. The kappa statistics for determining if processes of care were provided ranged from 0.56 to 0.83 and increased to 0.65 and 0.85 upon adjustment for the prevalence effect. Kappas for the composite quality indicators were lower, but improved with adjustment for the prevalence effect. The composite quality indicator with the highest adjusted kappa value was oxygenation assessment (0.93); the composite quality indicator with the lowest adjusted kappa value was antibiotic administration within 8 hours of hospital arrival (0.74). This study establishes the reliability of pneumonia indicators and underscores the need for reliability assessment at the quality indicator level, as well as at the component level.
为了证明评估整体质量指标可靠性（除了组件或变量层面的可靠性）的重要性，我们对四位图表摘要员在与肺炎相关的四个护理流程上的评分者间一致性进行了比较。356名医疗保险患者近期因肺炎出院的医院病历由不同的摘要员独立提取。使用卡帕值、患病率和偏差调整卡帕值、P（阳性）、P（阴性）以及偏差指数来评估综合质量指标及其组件的可靠性。用于确定接受资格以及推导与肺炎相关护理流程的数据元素的调整后卡帕值范围为0.68至1.0。与接受肺炎相关护理流程的总体资格相关的调整后卡帕值为0.63。用于确定是否提供护理流程的卡帕统计值范围为0.56至0.83，在对患病率效应进行调整后分别增至0.65和0.85。综合质量指标的卡帕值较低，但在对患病率效应进行调整后有所改善。调整后卡帕值最高的综合质量指标是氧合评估（0.93）；调整后卡帕值最低的综合质量指标是入院8小时内给予抗生素（0.74）。本研究确立了肺炎指标的可靠性，并强调了在质量指标层面以及组件层面进行可靠性评估的必要性。
