使用标准化视频病例评估医学沟通技巧：计算机客观结构化视频考试的可靠性

Using standardized video cases for assessment of medical communication skills: reliability of an objective structured video examination by computer.

作者信息

Hulsman R L, Mollema E D, Oort F J, Hoos A M, de Haes J C J M

机构信息

Department of Medical Psychology, J4, Academic Medical Centre, P.O. Box 22660, 1100 DD Amsterdam, The Netherlands.

出版信息

Patient Educ Couns. 2006 Jan;60(1):24-31. doi: 10.1016/j.pec.2004.11.010. Epub 2004 Dec 30.

DOI:10.1016/j.pec.2004.11.010

PMID:16332467

Abstract

OBJECTIVE

Using standardized video cases in a computerized objective structured video examination (OSVE) aims to measure cognitive scripts underlying overt communication behavior by questions on knowledge, understanding and performance. In this study the reliability of the OSVE assessment is analyzed using the generalizability theory.

METHODS

Third year undergraduate medical students from the Academic Medical Center of the University of Amsterdam answered short-essay questions on three video cases, respectively about history taking, breaking bad news, and decision making. Of 200 participants, 116 completed all three video cases. Students were assessed in three shifts, each using a set of parallel case editions. About half of all available exams were scored independently by two raters using a detailed rating manual derived from the other half. Analyzed were the reliability of the assessment, the inter-rater reliability, and interrelatedness of the three types of video cases and their parallel editions, by computing a generalizability coefficient G.

RESULTS

The test score showed a normal distribution. The students performed relatively well on the history taking type of video cases, and relatively poor on decision making and did relatively poor on the understanding ('knows why/when') type of questions. The reliability of the assessment was acceptable (G = 0.66). It can be improved by including up to seven cases in the OSVE. The inter-rater reliability was very good (G = 0.93). The parallel editions of the video cases appeared to be more alike (G = 0.60) than the three case types (G = 0.47).

DISCUSSION

The additional value of an OSVE is the differential picture that is obtained about covert cognitive scripts underlying overt communication behavior in different types of consultations, indicated by the differing levels of knowledge, understanding and performance. The validation of the OSVE score requires more research.

CONCLUSION AND PRACTICE IMPLICATIONS

A computerized OSVE has been successfully applied with third year undergraduate medical students. The test score meets psychometric criteria, enabling a proper discrimination between adequately and poorly performing students. The high inter-rater reliability indicates that a single rater is permitted.

摘要

目的

在计算机化客观结构化视频考试（OSVE）中使用标准化视频案例，旨在通过关于知识、理解和表现的问题来衡量公开交流行为背后的认知脚本。在本研究中，使用概化理论分析OSVE评估的可靠性。

方法

来自阿姆斯特丹大学学术医疗中心的三年级本科医学生回答了关于三个视频案例的短文问题，分别涉及病史采集、告知坏消息和决策制定。在200名参与者中，116人完成了所有三个视频案例。学生们分三个班次进行评估，每个班次使用一组平行的案例版本。大约一半的可用考试由两名评分者使用从另一半衍生而来的详细评分手册独立评分。通过计算概化系数G，分析了评估的可靠性、评分者间的可靠性以及三种类型视频案例及其平行版本的相关性。

结果

考试成绩呈正态分布。学生们在病史采集类型的视频案例上表现相对较好，在决策制定方面表现相对较差，在理解（“知道原因/时间”）类型的问题上表现也相对较差。评估的可靠性是可以接受的（G = 0.66）。通过在OSVE中纳入多达七个案例可以提高可靠性。评分者间的可靠性非常好（G = 0.93）。视频案例的平行版本似乎比三种案例类型更相似（G = 0.60）（G = 0.47）。