Center for Organization, Leadership and Management Research (COLMR), VA Boston Healthcare System, Boston, MA 02130, USA.
Med Care. 2012 Jan;50(1):74-85. doi: 10.1097/MLR.0b013e3182293edf.
The Agency for Healthcare Research and Quality (AHRQ) Patient Safety Indicators (PSIs) use administrative data to detect potentially preventable in-hospital adverse events. However, few studies have determined how accurately the PSIs identify true safety events.
We examined the criterion validity, specifically the positive predictive value (PPV), of 12 selected PSIs using clinical data abstracted from the Veterans Health Administration (VA) electronic medical record as the gold standard.
We identified PSI-flagged cases from 28 representative hospitals by applying the AHRQ PSI software (v.3.1a) to VA fiscal year 2003 to 2007 administrative data. Trained nurse-abstractors used standardized abstraction tools to review a random sample of flagged medical records (112 records per PSI) for the presence of true adverse events. Interrater reliability was assessed. We evaluated PPVs and associated 95% confidence intervals of each PSI and examined false positive (FP) cases to determine why they were incorrectly flagged and gain insight into how each PSI might be improved.
PPVs ranged from 28% (95% CI, 15%-43%) for Postoperative Hip Fracture to 87% (95% CI, 79%-92%) for Postoperative Wound Dehiscence. Common reasons for FPs included conditions that were present on admission (POA), coding errors, and lack of coding specificity. PSIs with the lowest PPVs had the highest proportion of FPs owing to POA.
Overall, PPVs were moderate for most of the PSIs. Implementing POA codes and using more specific ICD-9-CM codes would improve their validity. Our results suggest that additional coding improvements are needed before the PSIs evaluated herein are used for hospital reporting or pay for performance.
医疗保健研究与质量署（AHRQ）的患者安全指标（PSIs）使用行政数据来检测潜在可预防的院内不良事件。然而，很少有研究确定 PSIs 识别真实安全事件的准确性。
我们使用从退伍军人事务部（VA）电子病历中提取的临床数据作为金标准，检查了 12 个选定 PSI 的判定有效性，特别是阳性预测值（PPV）。
我们通过应用 AHRQ PSI 软件（v.3.1a）对 VA 财政年度 2003 年至 2007 年的行政数据，从 28 家有代表性的医院中确定 PSI 标记病例。经过培训的护士摘要员使用标准化的摘要工具，对标记病历（每个 PSI 112 份病历）进行随机抽样，以确定是否存在真实的不良事件。评估了组内可靠性。我们评估了每个 PSI 的 PPV 和相关的 95%置信区间，并检查了假阳性（FP）病例，以确定它们为什么被错误标记，并深入了解如何改进每个 PSI。
PPV 范围从术后髋关节骨折的 28%（95%CI，15%-43%）到术后伤口裂开的 87%（95%CI，79%-92%）。FP 的常见原因包括入院时存在的病症（POA）、编码错误和缺乏编码特异性。PPV 最低的 PSIs 由于 POA 导致 FP 的比例最高。
总体而言，大多数 PSI 的 PPV 为中等。实施 POA 代码并使用更具体的 ICD-9-CM 代码将提高它们的有效性。我们的结果表明，在本文评估的 PSIs 用于医院报告或绩效付费之前，还需要进行额外的编码改进。
