Center for Innovations in Quality, Effectiveness, and Safety, Michael E. DeBakey Veterans Affairs Medical Center, Houston, Texas.
Michael E. DeBakey Department of Surgery, Baylor College of Medicine, Houston, Texas.
JAMA Surg. 2023 Dec 1;158(12):1312-1319. doi: 10.1001/jamasurg.2023.4532.
Representative surgical case sampling, rather than universal review, is used by US Department of Veterans Affairs (VA) and private-sector national surgical quality improvement (QI) programs to assess program performance and to inform local QI and performance improvement efforts. However, it is unclear whether case sampling is robust for identifying hospitals with safety or quality concerns.
To evaluate whether the sampling strategy used by several national surgical QI programs provides hospitals with data that are representative of their overall quality and safety, as measured by 30-day mortality.
DESIGN, SETTING, AND PARTICIPANTS: This comparative effectiveness study was a national, hospital-level analysis of data from adult patients (aged ≥18 years) who underwent noncardiac surgery at a VA hospital between January 1, 2016, and September 30, 2020. Data were obtained from the VA Surgical Quality Improvement Program (representative sample) and the VA Corporate Data Warehouse surgical domain (100% of surgical cases). Data analysis was performed from July 1 to December 21, 2022.
The primary outcome was postoperative 30-day mortality. Quarterly, risk-adjusted, 30-day mortality observed-to-expected (O-E) ratios were calculated separately for each hospital using the sample and universal review cohorts. Outlier hospitals (ie, those with higher-than-expected mortality) were identified using an O-E ratio significantly greater than 1.0.
In this study of data from 113 US Department of Veterans Affairs hospitals, the sample cohort comprised 502 953 surgical cases and the universal review cohort comprised 1 703 140. The majority of patients in both the representative sample and the universal sample were men (90.2% vs 91.1%) and were White (74.7% vs 74.5%). Overall, 30-day mortality was 0.8% and 0.6% for the sample and universal review cohorts, respectively (P < .001). Over 2145 quarters of data, hospitals were identified as an outlier in 11.7% of quarters with sampling and in 13.2% with universal review. Average hospital quarterly 30-day mortality rates were 0.4%, 0.8%, and 0.9% for outlier hospitals identified using the sample only, universal review only, and concurrent identification in both data sources, respectively. For nonsampled cases, average hospital quarterly 30-day mortality rates were 1.0% at outlier hospitals and 0.5% at nonoutliers. Among outlier hospital quarters in the sample, 47.4% were concurrently identified with universal review. For those identified with universal review, 42.1% were concurrently identified using the sample.
In this national, hospital-level study, sampling strategies employed by national surgical QI programs identified less than half of hospitals with higher-than-expected perioperative mortality. These findings suggest that sampling may not adequately represent overall surgical program performance or provide stakeholders with the data necessary to inform QI efforts.
美国退伍军人事务部（VA）和私营部门国家手术质量改进（QI）计划使用代表性手术病例抽样，而不是普遍审查，来评估计划绩效，并为当地的 QI 和绩效改进工作提供信息。然而，目前尚不清楚病例抽样是否能够可靠地识别出存在安全或质量问题的医院。
评估几个国家手术 QI 计划使用的抽样策略是否为医院提供了与其整体质量和安全相关的数据，这是通过 30 天死亡率来衡量的。
设计、设置和参与者：这是一项全国性的、基于医院水平的分析，纳入了 2016 年 1 月 1 日至 2020 年 9 月 30 日期间在 VA 医院接受非心脏手术的成年患者（年龄≥18 岁）的数据。数据来自 VA 手术质量改进计划（代表性样本）和 VA 公司数据仓库手术领域（100%的手术病例）。数据分析于 2022 年 7 月 1 日至 12 月 21 日进行。
主要结果是术后 30 天死亡率。使用样本和普遍审查队列，分别为每个医院计算了季度风险调整后 30 天的观察到的与预期（O-E）比值。使用 O-E 比值显著大于 1.0 来识别死亡率高于预期的异常医院（即死亡率异常高的医院）。
在这项对来自 113 家美国退伍军人事务部医院的数据进行的研究中，样本队列包括 502953 例手术病例，普遍审查队列包括 1703140 例。代表性样本和普遍样本中的大多数患者都是男性（90.2%比 91.1%）和白人（74.7%比 74.5%）。总体而言，样本和普遍审查队列的 30 天死亡率分别为 0.8%和 0.6%（P<.001）。在 2145 个数据季度中，使用抽样方法识别出 11.7%的季度中有异常医院，使用普遍审查方法识别出 13.2%的季度中有异常医院。仅使用样本识别出的异常医院的平均医院季度 30 天死亡率为 0.4%，仅使用普遍审查识别出的异常医院的平均医院季度 30 天死亡率为 0.8%，同时在两个数据源中识别出的异常医院的平均医院季度 30 天死亡率为 0.9%。对于未抽样的病例，异常医院的平均医院季度 30 天死亡率为 1.0%，而非异常医院的死亡率为 0.5%。在样本中的异常医院季度中，47.4%同时被普遍审查识别。对于那些被普遍审查识别的医院，有 42.1%同时被样本识别。
在这项全国性的、基于医院水平的研究中，国家手术 QI 计划使用的抽样策略仅识别出不到一半的围手术期死亡率高于预期的医院。这些发现表明，抽样可能无法充分代表整体手术项目的绩效，或者无法为利益相关者提供必要的数据来指导 QI 工作。
