Gutierrez Marc, Wilson Kelsey, Bickford Brant, Yuhas Joseph, Markert Ronald, Burtson Kathryn M
Internal Medicine Program, Affiliated with Wright Patterson AFB and Wright State University, Wright-Patterson AFB, OH, USA.
Department of Internal Medicine and Neurology, Affiliated with Wright State University, Dayton, OH, USA.
J Med Educ Curric Dev. 2023 Oct 9;10:23821205231206058. doi: 10.1177/23821205231206058. eCollection 2023 Jan-Dec.
To determine whether incorporating our novel in-training evaluation report (ITER), which prompts each resident to list at least three self-identified learning goals, improved the quality of narrative assessments as measured by the Narrative Evaluation Quality Instrument (NEQI).
A total of 1468 narrative assessments from a single institution from 2017 to 2021 were deidentified, compiled, and sorted into the pre-intervention form arm and post-intervention form arm. Due to limitations in our residency management suite, incorporating learning goals required switching from an electronic form to a hand-deliver form. Comments were graded by two research personnel utilizing the NEQI's scale of 0-12, with 12 representing the maximum quality for a comment. The outcome of the study was the mean difference in NEQI score between the electronic pre-intervention period and paper post-intervention period.
The mean NEQI score for the pre-intervention period was 2.43 ± 3.34, and the mean NEQI score for the post-intervention period was 3.31 ± 1.71, with a mean difference of 0.88 (p < 0.001). In the pre-intervention period, 46% of evaluations were submitted without a narrative assessment (scored as a zero) while 1% of post-intervention period evaluations had no narrative assessment. Internal consistency reliability, as measured by Ebel's intraclass correlation coefficient (ICC), showed high agreement between the two raters (ICC = 0.92).
Our findings suggest that implementing a timely, hand-delivered paper ITER that incorporates resident learning goals can lead to overall higher-quality narrative assessments.
确定纳入我们新颖的培训期间评估报告（ITER）（该报告促使每位住院医师列出至少三个自我确定的学习目标）是否能提高叙事性评估的质量，该质量通过叙事性评估质量工具（NEQI）来衡量。
对2017年至2021年来自单一机构的1468份叙事性评估进行去识别、汇编，并分为干预前表格组和干预后表格组。由于我们住院医师管理套件的限制，纳入学习目标需要从电子表格切换为手工交付表格。两名研究人员使用NEQI的0至12分评分标准对评论进行评分，12分代表评论的最高质量。该研究的结果是干预前电子表格时期和干预后纸质表格时期NEQI评分的平均差异。
干预前时期的平均NEQI评分为2.43±3.34，干预后时期的平均NEQI评分为3.31±1.71，平均差异为0.88（p<0.001）。在干预前时期，46%的评估没有叙事性评估（评分为零），而干预后时期1%的评估没有叙事性评估。通过埃贝尔组内相关系数（ICC）衡量的内部一致性信度显示，两位评分者之间具有高度一致性（ICC = 0.92）。
我们的研究结果表明，实施一份及时的、手工交付的包含住院医师学习目标的纸质ITER可以带来整体质量更高的叙事性评估。
