Department of Pediatrics, Divisions of aGeneral Pediatrics.
Department of Pediatrics, Emory University School of Medicine and Division of Hospital Medicine, Children's Healthcare of Atlanta, Atlanta, Georgia.
Hosp Pediatr. 2024 Feb 1;14(2):137-145. doi: 10.1542/hpeds.2023-007418.
This study aimed to develop and evaluate an algorithm to reduce the chart review burden of improvement efforts by automatically labeling antibiotic selection as either guideline-concordant or -discordant based on electronic health record data for patients with community-acquired pneumonia (CAP).
We developed a 3-part algorithm using structured and unstructured data to assess adherence to an institutional CAP clinical practice guideline. The algorithm was applied to retrospective data for patients seen with CAP from 2017 to 2019 at a tertiary children's hospital. Performance metrics included positive predictive value (precision), sensitivity (recall), and F1 score (harmonized mean), with macro-weighted averages. Two physician reviewers independently assigned "actual" labels based on manual chart review.
Of 1345 patients with CAP, 893 were included in the training cohort and 452 in the validation cohort. Overall, the model correctly labeled 435 of 452 (96%) patients. Of the 286 patients who met guideline inclusion criteria, 193 (68%) were labeled as having received guideline-concordant antibiotics, 48 (17%) were labeled as likely in a scenario in which deviation from the clinical practice guideline was appropriate, and 45 (16%) were given the final label of "possibly discordant, needs review." The sensitivity was 0.96, the positive predictive value was 0.97, and the F1 was 0.96.
An automated algorithm that uses structured and unstructured electronic health record data can accurately assess the guideline concordance of antibiotic selection for CAP. This tool has the potential to improve the efficiency of improvement efforts by reducing the manual chart review needed for quality measurement.
本研究旨在开发并评估一种算法，通过使用电子健康记录数据，自动为社区获得性肺炎（CAP）患者的抗生素选择贴上符合或不符合指南的标签，从而减轻改善措施的病历审查负担。
我们使用结构化和非结构化数据开发了一个三部分算法，以评估对机构 CAP 临床实践指南的依从性。该算法应用于 2017 年至 2019 年在一家三级儿童医院就诊的 CAP 患者的回顾性数据。性能指标包括阳性预测值（精度）、敏感性（召回率）和 F1 分数（协调平均值），采用宏加权平均值。两名医生审查员独立根据手动病历审查分配“实际”标签。
在 1345 例 CAP 患者中，893 例被纳入训练队列，452 例被纳入验证队列。总体而言，该模型正确标记了 452 例中的 435 例（96%）。在符合指南纳入标准的 286 例患者中，193 例（68%）被标记为接受了符合指南的抗生素治疗，48 例（17%）被标记为在适当偏离临床实践指南的情况下可能符合指南，45 例（16%）被给予“可能不符合，需要审查”的最终标签。敏感性为 0.96，阳性预测值为 0.97，F1 为 0.96。
一种使用结构化和非结构化电子健康记录数据的自动算法可以准确评估 CAP 中抗生素选择的指南一致性。该工具有可能通过减少质量测量所需的手动病历审查，提高改善措施的效率。
