From the School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, Texas.
Department of Preventive and Restorative Dental Sciences, School of Dentistry, University of California San Francisco, San Francisco, California.
J Patient Saf. 2022 Aug 1;18(5):470-474. doi: 10.1097/PTS.0000000000000959.
To achieve high-quality health care, adverse events (AEs) must be proactively recognized and mitigated. However, there is often ambiguity in applying guidelines and definitions. We describe the iterative calibration process needed to achieve a shared definition of AEs in dentistry. Our alignment process includes both independent and consensus building approaches.
We explore the process of defining dental AEs and the steps necessary to achieve alignment across different care providers.
Teams from 4 dental institutions across the United States iteratively reviewed patient records after identification of charts using an automated trigger tool. Calibration across teams was supported through negotiated definition of AEs and standardization of evidence provided in review. Interrater reliability was assessed using descriptive and κ statistics.
After 5 iterative cycles of calibration, the teams (n = 8 raters) identified 118 cases. The average percent agreement for AE determination was 82.2%. Furthermore, the average, pairwise prevalence and bias-adjusted κ (PABAK) was 57.5% (κ = 0.575) for determining AE presence. The average percent agreement for categorization of the AE type was 78.5%, whereas the PABAK was 48.8%. Lastly, the average percent agreement for categorization of AE severity was 82.2% and the corresponding PABAK was 71.7%.
Successful calibration across reviewers is possible after consensus building procedures. Higher levels of agreement were found when categorizing severity (of identified events) rather than the events themselves. Our results demonstrate the need for collaborative procedures as well as training for the identification and severity rating of AEs.
为了实现高质量的医疗保健，必须主动识别和减轻不良事件 (AE)。然而，在应用指南和定义时常常存在歧义。我们描述了在牙科中实现 AE 共享定义所需的迭代校准过程。我们的对齐过程包括独立和共识构建方法。
我们探讨了定义牙科 AE 的过程以及实现不同医疗保健提供者之间一致性所需的步骤。
来自美国 4 家牙科机构的团队使用自动触发工具在识别图表后对患者记录进行迭代审查。通过协商确定 AE 的定义和审查中提供证据的标准化，支持团队之间的校准。使用描述性和κ统计数据评估了组内一致性。
经过 5 轮迭代校准，团队 (n = 8 名评估者) 确定了 118 例病例。AE 确定的平均百分比一致性为 82.2%。此外，AE 存在的平均、成对流行率和偏差调整 κ（PABAK）为 57.5%（κ = 0.575）。AE 类型分类的平均百分比一致性为 78.5%，而 PABAK 为 48.8%。最后，AE 严重程度分类的平均百分比一致性为 82.2%，相应的 PABAK 为 71.7%。
在共识建立程序之后，对审核员进行成功的校准是可能的。在对已识别事件的严重程度进行分类时，发现一致性更高，而不是对事件本身进行分类。我们的结果表明需要协作程序以及对 AE 的识别和严重程度评级进行培训。
