Abrams Mary Ann, Crichton Kristin Garton, Oberle Edward J, Flowers Stacy, Crawford Timothy N, Perry Michael F, Mahan John D, Reed Suzanne
Health Lit Res Pract. 2023 Oct;7(4):e187-e196. doi: 10.3928/24748307-20230919-01. Epub 2023 Oct 5.
Teach Back (TB) is recommended to assess and ensure patient understanding, thereby promoting safety, quality, and equity. There are many TB trainings, typically lacking assessment tools with validity evidence. We used a pediatric resident competency-based communication curriculum to develop initial validity evidence and refinement recommendations for a Teach-back Observation Tool (T-BOT).
This study aimed to develop initial validity evidence for a refined T-BOT and provide guidance for further enhancements to improve essential TB skills training among pediatric residents.
After an interactive health literacy (HL) training, residents participated in recorded standardized patient (SP) encounters. Raters developed T-BOT scoring criteria, then scored a gold standard TB video and resident SP encounters. For agreement, Fleiss' Kappa was computed for >2 raters, and Cohen's Kappa for two raters. Percent agreement and intraclass correlation (ICC) were calculated. Statistics were calculated for gold standard (GS) and TB items overall for all six raters, and for five faculty raters. Agreement was based on Kappa: no agreement (≤0), none to slight (0.01-0.20), fair (0.21-0.40), moderate (0.41-0.60), substantial (0.61-0.80), almost perfect (0.81-1.00).
For six raters, Kappa for the GS was 0.554 (moderate agreement) with 71.4% agreement; I = .597; for SP encounters, it was 0.637 (substantial) with 65.4% agreement; I = .647. Individual item agreement for SP encounters average was 0.605 (moderate), ranging from 0.142 (slight) to 1 (perfect). For five faculty raters, Kappa for the GS was 0.779 (substantial) with 85.7% agreement; I = .824; for resident SP encounters, it was 0.751 (substantial), with 76.9% agreement; I = .759. Individual item agreement on SP encounters average was 0.718 (substantial), ranging from 0.156 (slight) to 1 (perfect).
We provide initial validity evidence for a modified T-BOT and recommendations for improvement. With further refinements to increase validity evidence, accompanied by shared understanding of TB and rating criteria, the T-BOT may be useful in strengthening approaches to teaching and improving essential TB skills among health care team members, thereby increasing organizational HL and improving outcomes. [].
推荐采用反馈教学法（TB）来评估并确保患者理解，从而提高安全性、质量和公平性。有许多TB培训，但通常缺乏具有效度证据的评估工具。我们利用一个基于儿科住院医师胜任力的沟通课程，为反馈教学观察工具（T-BOT）制定初步的效度证据和完善建议。
本研究旨在为改进后的T-BOT制定初步的效度证据，并为进一步改进提供指导，以加强儿科住院医师的基本TB技能培训。
在进行交互式健康素养（HL）培训后，住院医师参与录制的标准化患者（SP）问诊。评分者制定T-BOT评分标准，然后对一个金标准TB视频和住院医师的SP问诊进行评分。对于一致性，计算>2名评分者的Fleiss' Kappa以及两名评分者的Cohen's Kappa。计算百分比一致性和组内相关系数（ICC）。计算所有六名评分者以及五名教员评分者对金标准（GS）和TB项目总体的统计数据。一致性基于Kappa：无一致性（≤0）、无至轻微（0.01 - 0.20）、一般（0.21 - 0.40）、中等（0.41 - 0.60）、显著（0.61 - 0.80）、几乎完美（0.81 - 1.00）。
对于六名评分者，GS的Kappa为0.554（中等一致性），一致性为71.4%；I = 0.597；对于SP问诊，Kappa为0.637（显著），一致性为65.4%；I = 0.647。SP问诊单个项目的平均一致性为0.605（中等），范围从0.142（轻微）到1（完美）。对于五名教员评分者，GS的Kappa为0.779（显著），一致性为85.7%；I = 0.824；对于住院医师的SP问诊，Kappa为0.
