Suppr超能文献

医学生基于反思性写作的学习档案袋评估工具的结构效度

Construct Validity of an Instrument for Assessment of Reflective Writing-Based Portfolios of Medical Students.

作者信息

Kassab Salah Eldin, Bidmos Mubarak, Nomikos Michail, Daher-Nashif Suhad, Kane Tanya, Sarangi Srikant, Abu-Hijleh Marwan

机构信息

Department of Basic Medical Sciences, College of Medicine, QU Health, Qatar University, Doha, Qatar.

Department of Population Medicine, College of Medicine, Qatar University, QU Health, Doha, Qatar.

出版信息

Adv Med Educ Pract. 2020 Jun 3;11:397-404. doi: 10.2147/AMEP.S256338. eCollection 2020.

Abstract

PURPOSE

Assessment of reflective writing for medical students is challenging, and there is lack of an available instrument with good psychometric properties. The authors developed a new instrument for assessment of reflective writing-based portfolios and examined the construct validity of this instrument.

METHODS

After an extensive literature review and pilot testing of the instrument, two raters assessed the reflective writing-based portfolios from years 2 and 3 medical students (n=135) on three occasions. The instrument consists of three criteria: organization, description of an experience and reflection on the experience. We calculated the reliability of scores using generalizability theory with a fully crossed design and two facets (raters and occasions). In addition, we measured criterion validity by testing correlations with students' scores using other assessment methods.

RESULTS

The dependability (Φ) coefficient of the portfolio scores was 0.75 using two raters on three occasions. Students' portfolio scores represented 46.6% of the total variance across all score comparisons. The variance due to occasions was negligible, while the student-occasion interaction was small. The variance due to student-rater interaction represented 17.7%, and the remaining 27.7% of the variance was due to unexplained sources of error. The decision (D) study suggested that an acceptable dependability (Φ = 0.70 and 0.72) can be achieved by using two raters for one and two occasions, respectively. Finally, we found moderate to large effect-size correlations between students' scores in reflective writing-based portfolios and communication skills (r = 0.47) and PBL tutorials (r = 0.50).

CONCLUSION

We demonstrated the presence of different sources of evidence that support construct validity of the study instrument. Further studies are warranted before utilizing this instrument for summative assessment of students' reflective writing-based portfolios in other medical schools.

摘要

目的

对医学生的反思性写作进行评估具有挑战性,且缺乏一种具有良好心理测量特性的可用工具。作者开发了一种用于评估基于反思性写作的档案袋的新工具,并检验了该工具的结构效度。

方法

在对该工具进行广泛的文献综述和预测试后,两名评分者对二、三年级医学生(n = 135)基于反思性写作的档案袋进行了三次评估。该工具包括三个标准:组织、经验描述和对经验的反思。我们使用完全交叉设计和两个方面(评分者和评估次数)的概化理论计算分数的可靠性。此外,我们通过测试与学生使用其他评估方法所得分数的相关性来测量效标效度。

结果

使用两名评分者在三个时间点对档案袋分数进行评估,其可靠性(Φ)系数为0.75。在所有分数比较中,学生的档案袋分数占总方差的46.6%。评估次数导致的方差可忽略不计,而学生 - 评估次数的交互作用较小。评分者 - 学生交互作用导致的方差占17.7%,其余27.7%的方差归因于无法解释的误差来源。决策(D)研究表明,分别使用两名评分者进行一次和两次评估,可实现可接受的可靠性(Φ = 0.70和0.72)。最后,我们发现基于反思性写作的档案袋中学生分数与沟通技能(r = 0.47)和基于问题的学习辅导(r = 0.50)之间存在中度到高度的效应量相关性。

结论

我们证明了存在不同来源的证据支持该研究工具的结构效度。在其他医学院校将该工具用于对学生基于反思性写作的档案袋进行总结性评估之前,有必要进行进一步研究。

https://cdn.ncbi.nlm.nih.gov/pmc/blobs/ee95/7276316/e37b400083ca/AMEP-11-397-g0001.jpg

文献AI研究员

20分钟写一篇综述,助力文献阅读效率提升50倍。

立即体验

用中文搜PubMed

大模型驱动的PubMed中文搜索引擎

马上搜索

文档翻译

学术文献翻译模型,支持多种主流文档格式。

立即体验