HTRG - Heidelberg Trauma Research Group, Center for Orthopedics, Trauma Surgery and Spinal Cord Injury, Trauma and Reconstructive Surgery, Heidelberg University Hospital, Schlierbacher Landstrasse 200a, D-69118, Heidelberg, Germany.
Department of General, Visceral and Transplantation Surgery Heidelberg University Hospital, D-69120, Heidelberg, Germany.
BMC Med Educ. 2018 Dec 27;18(1):320. doi: 10.1186/s12909-018-1430-9.
There is an increasing need for objective and validated educational concepts. This holds especially true for surgical procedures like chest tube insertion (CTI). Thus, we developed an instrument for objectification of learning successes: the assessment scale based on Objective Structured Assessment of Technical Skill (OSATS) for chest tube insertion, which is evaluated in this study. Primary endpoint was the evaluation of intermethod reliability (IM). Secondary endpoints are 'indirect' interrater reliability (IR) and construct validity of the scale (CV).
Every participant (N = 59) performed a CTI on a porcine thorax. Participants received three ratings (one 'direct' on site, two 'indirect' via video rating). IM compares 'direct' with 'indirect' ratings. IR was assessed between 'indirect' ratings. CV was investigated by subgroup analysis based on prior experience in CTI for 'direct' and 'indirect' rating.
We included 59 medical students to our study. IM showed moderate conformity ('direct' vs. 'indirect 1' ICC = 0.735, 95% CI: 0.554-0.843; 'direct' vs. 'indirect 2' ICC = 0.722, 95% CI 0.533-0.835) and good conformity between 'direct' vs. 'average indirect' rating (ICC = 0.764, 95% CI: 0.6-0.86). IR showed good conformity (ICC = 0.84, 95% CI: 0.707-0.91). CV was proven between subgroups in 'direct' (p = 0.037) and 'indirect' rating (p = 0.013).
Results for IM suggest equivalence for 'direct' and 'indirect' ratings, while both IR and CV was demonstrated in both rating methods. Thus, the assessment scale seems a reliable method for rating trainees' performances 'directly' as well as 'indirectly'. It may help to objectify and facilitate the assessment of training of chest tube insertion.
对于像胸腔引流管插入术（CTI）这样的手术程序，客观和经过验证的教育概念的需求越来越大。因此，我们开发了一种用于实现学习成果客观化的工具：基于客观结构化临床技能评估（OSATS）的胸腔引流管插入评估量表，本研究对其进行了评估。主要终点是评估方法间可靠性（IM）。次要终点是量表的间接评分者间可靠性（IR）和结构有效性（CV）。
每位参与者（N=59）在猪的胸腔上进行 CTI。参与者接受三种评分（一种是现场的“直接”评分，两种是通过视频的“间接”评分）。IM 将“直接”与“间接”评分进行比较。IR 通过“间接”评分之间的比较进行评估。CV 通过基于 CTI 先前经验的亚组分析进行调查，用于“直接”和“间接”评分。
我们纳入了 59 名医学生进行研究。IM 显示中度一致性（“直接”与“间接 1”ICC=0.735，95%CI：0.554-0.843；“直接”与“间接 2”ICC=0.722，95%CI 0.533-0.835），“直接”与平均“间接”评分之间的一致性良好（ICC=0.764，95%CI：0.6-0.86）。IR 显示出良好的一致性（ICC=0.84，95%CI：0.707-0.91）。在“直接”（p=0.037）和“间接”（p=0.013）评分中，CV 在亚组间得到证实。
IM 的结果表明“直接”和“间接”评分具有等效性，而 IR 和 CV 均在两种评分方法中得到证实。因此，该评估量表似乎是一种可靠的方法，可用于直接和间接评估学员的表现。它可以帮助客观化和促进胸腔引流管插入术的培训评估。
