Division of Pediatric Surgery, Department of Surgery, Indiana University School of Medicine, Indianapolis, Indiana.
Department of Surgery, Indiana University School of Medicine, Center for Outcomes Research in Surgery, Indianapolis, Indiana.
J Surg Res. 2019 Jan;233:82-87. doi: 10.1016/j.jss.2018.07.045. Epub 2018 Aug 17.
The Congenital Diaphragmatic Hernia Study Group (CDHSG) registry is a multi-institutional tool to track outcomes of patients with CDH. The CDHSG asks surgeons to categorize diaphragmatic defect sizes as type A-D based on published guidelines. The reported size of the defect has been correlated with patient outcomes, but the reliability of this system has never been studied. Our goal was to evaluate the inter- and intra-rater reliability of the CDHSG grading system.
Forty-six operative notes from CDH patients that underwent surgical repair at a single institution were collected and cropped to include only the information necessary to grade the hernia defect based on the CDHSG guidelines. The defects were graded by nine pediatric surgeons on two separate occasions (18 wk apart). Inter-rater reliability was calculated using a Cohen's kappa (κ). Intra-rater reliability was calculated using an intraclass correlation coefficient.
Inter-rater reliability was minimal to weak (κ = 0.395, κ = 0.424). Agreement ranged from 19.57% (κ = -0.0745) to 82.61% (κ = 0.7543). Inter-rater agreement was similar despite operative findings and outcomes: survival yes/no (κ = 0.3690, κ = 0.3518), need for ECMO yes/no (κ = 0.3323, κ = 0.3362), patch repair yes/no (κ = 0.2050, κ = 0.1916), and liver up/down (κ = 0.2941, κ = 0.4404). Intra-rater reliability was good to excellent (intraclass correlation coefficient = 0.88, 95% CI [0.83-0.92]). Agreement with oneself ranged from 71.74% to 93.48%.
The demonstrated weak inter-rater reliability of the current CDHSG grading system shows the need for improvement in how the grading system is applied by surgeons when reporting CDH defect size.
先天性膈疝研究组（CDHSG）登记处是一个多机构工具，用于跟踪 CDH 患者的结局。CDHSG 要求外科医生根据已发表的指南将膈疝缺损大小分类为 A-D 型。报道的缺损大小与患者结局相关，但该系统的可靠性从未得到研究。我们的目标是评估 CDHSG 分级系统的组内和组间可靠性。
收集了一家机构接受手术修复的 46 例 CDH 患者的 46 份手术记录，并裁剪为仅包含根据 CDHSG 指南对疝缺损进行分级所需的信息。9 名儿科外科医生在两次不同的时间（相隔 18 周）对缺陷进行分级。使用 Cohen's kappa（κ）计算组间可靠性。使用组内相关系数计算组内可靠性。
组间可靠性为弱至中度（κ=0.395，κ=0.424）。一致性范围从 19.57%（κ=-0.0745）到 82.61%（κ=0.7543）。尽管手术结果和结局不同，但组间一致性相似：生存与否（κ=0.3690，κ=0.3518），是否需要 ECMO（κ=0.3323，κ=0.3362），是否修补补丁（κ=0.2050，κ=0.1916），肝脏上/下（κ=0.2941，κ=0.4404）。组内可靠性为好至优（组内相关系数=0.88，95%CI [0.83-0.92]）。与自己的一致性范围从 71.74%到 93.48%。
当前 CDHSG 分级系统显示出较弱的组间可靠性，表明外科医生在报告 CDH 缺陷大小时需要改进分级系统的应用方式。
