Ban Kristen A, Cohen Mark E, Ko Clifford Y, Friedberg Mark W, Stulberg Jonah J, Zhou Lynn, Hall Bruce L, Hoyt David B, Bilimoria Karl Y
*American College of Surgeons, Chicago, IL †Department of Surgery, Loyola University Medical Center, Maywood, IL ‡Department of Surgery, University of California Los Angeles, Los Angeles, CA §RAND Corporation, Boston, MA ¶Brigham and Women's Hospital, Harvard Medical School, Boston, MA ||Surgical Outcomes and Quality Improvement Center, Department of Surgery, Feinberg School of Medicine, Northwestern University, Chicago, IL **Department of Surgery, Washington University in St. Louis, St. Louis, MO ††Center for Health Policy and the Olin Business School at Washington University in St. Louis, St. Louis, MO ‡‡John Cochran Veterans Affairs Medical Center, St. Louis, MO §§BJC Healthcare, St. Louis, MO.
Ann Surg. 2016 Oct;264(4):566-74. doi: 10.1097/SLA.0000000000001858.
The ProPublica Surgeon Scorecard is the first nationwide, multispecialty public reporting of individual surgeon outcomes. However, ProPublica's use of a previously undescribed outcome measure (composite of in-hospital mortality or 30-day related readmission) and inclusion of only inpatients have been questioned. Our objectives were to (1) determine the proportion of cases excluded by ProPublica's specifications, (2) assess the proportion of inpatient complications excluded from ProPublica's measure, and (3) examine the validity of ProPublica's outcome measure by comparing performance on the measure to well-established postoperative outcome measures.
Using ACS-NSQIP data (2012-2014) for 8 ProPublica procedures and for All Operations, the proportion of cases meeting all ProPublica inclusion criteria was determined. We assessed the proportion of complications occurring inpatient, and thus not considered by ProPublica's measure. Finally, we compared risk-adjusted performance based on ProPublica's measure specifications to established ACS-NSQIP outcome measure performance (eg, death/serious morbidity, mortality).
ProPublica's inclusion criteria resulted in elimination of 82% of all operations from assessment (range: 42% for total knee arthroplasty to 96% for laparoscopic cholecystectomy). For all ProPublica operations combined, 84% of complications occur during inpatient hospitalization (range: 61% for TURP to 88% for total hip arthroplasty), and are thus missed by the ProPublica measure. Hospital-level performance on the ProPublica measure correlated weakly with established complication measures, but correlated strongly with readmission (R = 0.834, P < 0.001).
ProPublica's outcome measure specifications exclude 82% of cases, miss 84% of postoperative complications, and correlate poorly with well-established postoperative outcomes. Thus, the validity of the ProPublica Surgeon Scorecard is questionable.
ProPublica外科医生记分卡是首个全国性、多专业的个体外科医生手术结果公开报告。然而,ProPublica使用了一种此前未描述的结果指标(住院死亡率或30天相关再入院的综合指标)且仅纳入了住院患者,这受到了质疑。我们的目的是:(1)确定被ProPublica的标准排除在外的病例比例;(2)评估被ProPublica的指标排除在外的住院并发症比例;(3)通过将该指标的表现与成熟的术后结果指标进行比较,检验ProPublica结果指标的有效性。
使用美国外科医师学会国家外科质量改进计划(ACS-NSQIP)2012 - 2014年8种ProPublica手术及所有手术的数据,确定符合所有ProPublica纳入标准的病例比例。我们评估了住院期间发生的并发症比例,因此这些并发症未被ProPublica的指标考虑在内。最后,我们将基于ProPublica指标标准的风险调整表现与已确立的ACS-NSQIP结果指标表现(如死亡/严重并发症、死亡率)进行比较。
ProPublica的纳入标准导致82%的所有手术被排除在评估之外(范围:全膝关节置换术为42%,腹腔镜胆囊切除术为96%)。对于所有ProPublica手术合并计算,84%的并发症发生在住院期间(范围:经尿道前列腺切除术为61%,全髋关节置换术为88%),因此被ProPublica的指标遗漏。医院层面在ProPublica指标上的表现与已确立的并发症指标相关性较弱,但与再入院相关性很强(R = 0.834,P < 0.001)。
ProPublica的结果指标标准排除了82%的病例,遗漏了84%的术后并发症,且与成熟的术后结果相关性较差。因此,ProPublica外科医生记分卡的有效性值得怀疑。