From the Department of Radiology, University Hospital Basel, Petersgraben 4, 4031 Basel, Switzerland.
Radiology. 2024 Sep;312(3):e233065. doi: 10.1148/radiol.233065.
Background Report writing skills are a core competency to be acquired during residency, yet objective tools for tracking performance are lacking. Purpose To investigate whether the Jaccard index, derived from report comparison, can objectively illustrate learning curves in report writing performance throughout radiology residency. Materials and Methods Retrospective data from 246 984 radiology reports written from September 2017 to November 2022 in a tertiary care radiology department were included. Reports were scored using the Jaccard similarity coefficient (ie, a quantitative expression of the amount of edits performed; range, 0-1) of residents' draft (unsupervised initial attempt at a complete report) or preliminary reports (following joint readout with attending physicians) and faculty-reviewed final reports. Weighted mean Jaccard similarity was compared between years of experience using Welch analysis of variance with post hoc testing overall, per imaging division, and per modality. Relationships with years and quarters of resident experience were assessed using Spearman correlation. Results This study included 53 residents (mean report count, 4660 ± 3546; 1-5 years of experience). Mean Jaccard similarity of preliminary reports increased by 6% from 1st-year to 5th-year residents (0.86 ± 0.22 to 0.92 ± 0.15; < .001). Spearman correlation demonstrated a strong relationship between residents' experience and higher report similarity when aggregated for years ( = 0.99 [95% CI: 0.85, 1.00]; < .001) or quarters of experience ( = 0.90 [95% CI: 0.73, 0.96]; < .001). For residents' draft reports, Jaccard similarity increased by 14% over the course of the 5-year residency program (0.68 ± 0.27 to 0.82 ± 0.23; < .001). Subgroup analysis confirmed similar trends for all imaging divisions and modalities (eg, in musculoskeletal imaging, from 0.77 ± 0.31 to 0.91 ± 0.16 [ < .001]; = 0.98 [95% CI: 0.72, 1.00] [ < .001]). Conclusion Residents' report writing performance increases with experience. Trends can be quantified with the Jaccard index, with a 6% improvement from 1st- to 5th-year residents, indicating its effectiveness as a tool for evaluating training progress and guiding education over the course of residency. © RSNA, 2024 See also the editorial by Bruno in this issue.
背景 撰写背景报告是住院医师必须掌握的一项核心技能，但目前缺乏用于跟踪绩效的客观工具。目的 旨在探讨报告比较得出的杰卡德指数是否可以客观地说明放射科住院医师在报告撰写表现方面的学习曲线。材料与方法 本研究回顾性纳入了 2017 年 9 月至 2022 年 11 月期间在一家三级保健放射科完成的 246984 份放射学报告，这些报告来自 246 名住院医师。使用杰卡德相似系数（即对执行的编辑量的定量表达；范围，0-1）对住院医师的草稿（未经监督的完成报告的初次尝试）或初步报告（与主治医生联合阅读后）和由教员审查的最终报告进行评分。使用方差分析的 Welch 检验和事后检验，比较不同年限经验的住院医师的加权平均杰卡德相似系数，整体比较、按影像科室比较和按模态比较。使用 Spearman 相关分析评估与住院医师经验年限和季度的关系。结果 本研究共纳入 53 名住院医师（平均报告数量为 4660±3546；1-5 年工作经验）。从第 1 年到第 5 年，住院医师初步报告的杰卡德相似系数提高了 6%（0.86±0.22 至 0.92±0.15；<0.001）。Spearman 相关分析表明，当将经验年限或季度汇总时，住院医师的经验与更高的报告相似度之间存在很强的关系（=0.99[95%置信区间：0.85，1.00]；<0.001）。对于住院医师的草稿报告，在 5 年住院医师培训期间，杰卡德相似系数提高了 14%（0.68±0.27 至 0.82±0.23；<0.001）。亚组分析证实了所有影像科室和模态的相似趋势（例如，在肌肉骨骼成像中，从 0.77±0.31 提高到 0.91±0.16[<0.001]；=0.98[95%置信区间：0.72，1.00]；<0.001）。结论 住院医师的报告撰写表现随经验的增加而提高。趋势可以使用杰卡德指数进行量化，第 1 年至第 5 年的住院医师提高了 6%，这表明其作为评估培训进展和指导住院医师培训期间教育的工具的有效性。©RSNA，2024 另请参阅本期 Bruno 的社论。
