Dr. Lockspeiser is assistant professor, Department of Pediatrics, University of Colorado School of Medicine, Aurora, Colorado. Ms. Schmitter is program manager for medical education, Department of Pediatrics, University of Colorado School of Medicine, Aurora, Colorado. Dr. Lane is professor and vice chair for medical education, Department of Pediatrics, University of Colorado School of Medicine, Aurora, Colorado. Dr. Hanson is professor and director of medical education research and development, Departments of Pediatrics and Family Medicine, University of Colorado School of Medicine, Aurora, Colorado. Dr. Rosenberg is professor and director, Pediatric Residency Program, Department of Pediatrics, University of Colorado School of Medicine, Aurora, Colorado. Dr. Park is assistant professor, Department of Medical Education, University of Illinois at Chicago College of Medicine, Chicago, Illinois.
Acad Med. 2013 Oct;88(10):1558-63. doi: 10.1097/ACM.0b013e3182a352e6.
To provide validity evidence for use of the Learning Goal Scoring Rubric to assess the quality of written learning goals and residents' goal writing skills.
This two-part study used the rubric to assess University of Colorado third-year pediatric residents' written learning goals to obtain validity evidence. In study 1, five raters independently scored 48 goals written in 2010-2011 and 2011-2012 by 48 residents, who also responded to the Jefferson Scale of Physician Lifelong Learning (JeffSPLL). In study 2, two raters independently scored 48 goals written in 2011-2012 by 12 residents. Intraclass correlation coefficients (ICCs) assessed rater agreement to provide evidence for response process. Generalizability theory assessed internal structure. Independent-samples Mann-Whitney U tests and correlations assessed relationship to other variables. Content was matched to published literature and instructional methods.
The ICC was 0.71 for the overall rubric. In study 1, where the generalizability study's (G study's) object of measurement was learning goals, the phi coefficient was 0.867. In study 2, where the G study's object of measurement was the resident (goal writing skill), the phi coefficient was 0.751. The total mean score of residents with goal writing training was significantly higher than that of those without (7.54 versus 4.98, P < .001). Correlation between goal quality and JeffSPLL score was not significant. Investigators agreed that the content matched the published literature and instructional methods.
Preliminary validity evidence indicates that this scoring rubric can assess learning goal quality and goal writing skill.
为使用学习目标评分细则评估书面学习目标的质量和住院医师目标写作技能提供有效性证据。
这项两部分的研究使用评分细则评估科罗拉多大学三年级儿科住院医师在 2010-2011 年和 2011-2012 年期间撰写的 48 份学习目标，以获得有效性证据。在研究 1 中，5 名评分者独立对 48 名住院医师在 2010-2011 年和 2011-2012 年期间撰写的 48 份目标进行评分，住院医师还对杰斐逊医生终身学习量表（JeffSPLL）进行了回应。在研究 2 中，2 名评分者独立对 12 名住院医师在 2011-2012 年期间撰写的 48 份目标进行评分。组内相关系数（ICC）评估评分者的一致性，为反应过程提供证据。概化理论评估内部结构。独立样本 Mann-Whitney U 检验和相关性评估与其他变量的关系。内容与已发表的文献和教学方法相匹配。
整个评分细则的 ICC 为 0.71。在研究 1 中，一般化研究（G 研究）的测量对象是学习目标，phi 系数为 0.867。在研究 2 中，G 研究的测量对象是住院医师（目标写作技能），phi 系数为 0.751。接受过目标写作培训的住院医师的总平均得分明显高于未接受培训的住院医师（7.54 分比 4.98 分，P<.001）。目标质量与 JeffSPLL 得分之间的相关性不显著。研究人员一致认为内容与已发表的文献和教学方法相匹配。
初步有效性证据表明，该评分细则可评估学习目标质量和目标写作技能。
