基于美国医师执照考试新的病历书写格式的病历评分细则的有效性证据。

Validity evidence for a patient note scoring rubric based on the new patient note format of the United States Medical Licensing Examination.

机构信息

Dr. Park is assistant professor, Department of Medical Education, University of Illinois at Chicago College of Medicine, Chicago, Illinois. Dr. Lineberry is assistant professor, Department of Medical Education, University of Illinois at Chicago College of Medicine, Chicago, Illinois. Dr. Hyderi is associate dean for curriculum and associate professor, Department of Family Medicine, University of Illinois at Chicago College of Medicine, Chicago, Illinois. Dr. Bordage is professor, Department of Medical Education, University of Illinois at Chicago College of Medicine, Chicago, Illinois. Dr. Riddle is director of faculty development and research assistant professor, Department of Medical Education, University of Illinois at Chicago College of Medicine, Chicago, Illinois. Dr. Yudkowsky is director, Graham Clinical Performance Center, and associate professor, Department of Medical Education, University of Illinois at Chicago College of Medicine, Chicago, Illinois.

出版信息

Acad Med. 2013 Oct;88(10):1552-7. doi: 10.1097/ACM.0b013e3182a34b1e.

DOI:10.1097/ACM.0b013e3182a34b1e

PMID:23969362

Abstract

PURPOSE

This study examines validity evidence for the Patient Note Scoring Rubric, which was developed for a local graduation competency exam (GCE) to assess patient notes written in the new United States Medical Licensing Examination (USMLE) Step 2 Clinical Skills format. The rubric was designed to measure three dimensions: Documentation, justified differential diagnosis (DDX), and Workup.

METHOD

Analyses used GCE data from 170 fourth-year medical students who completed five standardized patient (SP) cases in May 2012. Five physician raters each scored all responses for one case. Internal structure was examined using correlations between dimensions and between cases; a generalizability study was also conducted. Relationship to other variables was examined by correlating patient note scores with SP encounter scores. Consequence was assessed by comparing pass-fail rates between the rubric and the previous global rating. Response process was examined using rater feedback.

RESULTS

Correlations between scores from different dimensions ranged between 0.33 and 0.44. Reliability of scores based on the phi coefficient was 0.43; 15 cases were required to reach a phi coefficient of 0.70. Evidence of case specificity was found. Documentation scores were moderately correlated with SP scores for data gathering (r = 0.47, P < .001). There was no meaningful change in pass-fail rates. Raters' feedback indicated that they required more training for scoring the DDX and Workup dimensions.

CONCLUSIONS

There is initial validity evidence for use of this rubric to score local clinical exams that are based on the new USMLE patient note format.

摘要

目的

本研究旨在检验专为本地毕业综合考试（GCE）开发的患者笔记评分量表的效度证据，该考试用于评估以新的美国医师执照考试（USMLE）第二阶段临床技能格式书写的患者笔记。该量表旨在衡量三个维度：记录、合理的鉴别诊断（DDX）和检查。

方法

使用 2012 年 5 月 170 名四年级医学生参加的 GCE 数据进行分析，这些学生完成了五个标准化患者（SP）病例。五名医生评分者对每个病例的所有回答进行评分。通过维度之间和病例之间的相关性来检查内部结构；还进行了一项可概括性研究。通过将患者笔记评分与 SP 遭遇评分相关联，来检查与其他变量的关系。通过比较量表和之前的整体评分之间的通过/失败率来评估后果。通过评分者的反馈来检查反应过程。

结果

来自不同维度的评分之间的相关性在 0.33 到 0.44 之间。基于 phi 系数的评分可靠性为 0.43；需要 15 个病例才能达到 phi 系数为 0.70。发现了案例特异性的证据。记录评分与 SP 评分在数据收集方面呈中度相关（r = 0.47，P <.001）。通过/失败率没有明显变化。评分者的反馈表明，他们需要更多的培训来评分 DDX 和检查维度。

结论

有初步的效度证据表明，该量表可用于评分基于新 USMLE 患者笔记格式的本地临床考试。

相似文献

Validity evidence for a patient note scoring rubric based on the new patient note format of the United States Medical Licensing Examination.基于美国医师执照考试新的病历书写格式的病历评分细则的有效性证据。

Acad Med. 2013 Oct;88(10):1552-7. doi: 10.1097/ACM.0b013e3182a34b1e.

Inter-rater reliability and generalizability of patient note scores using a scoring rubric based on the USMLE Step-2 CS format.使用基于美国医师执照考试第二步临床技能考试（USMLE Step-2 CS）格式的评分标准时，评分者间信度及患者记录分数的可推广性。

Adv Health Sci Educ Theory Pract. 2016 Oct;21(4):761-73. doi: 10.1007/s10459-015-9664-3. Epub 2016 Jan 12.

Differential Weighting for Subcomponent Measures of Integrated Clinical Encounter Scores Based on the USMLE Step 2 CS Examination: Effects on Composite Score Reliability and Pass-Fail Decisions.基于美国医师执照考试第二步临床技能考试的综合临床问诊分数子成分测量的差异加权：对综合分数可靠性及通过-未通过决策的影响

Acad Med. 2016 Nov;91(11 Association of American Medical Colleges Learn Serve Lead: Proceedings of the 55th Annual Research in Medical Education Sessions):S24-S30. doi: 10.1097/ACM.0000000000001359.

Assessing the impact of modifications to the documentation component's scoring rubric and rater training on USMLE integrated clinical encounter scores.评估对文档部分评分细则和评分员培训的修改对 USMLE 综合临床考试分数的影响。

Acad Med. 2009 Oct;84(10 Suppl):S97-100. doi: 10.1097/ACM.0b013e3181b361d4.

Evaluating construct equivalence and criterion-related validity for repeat examinees on a standardized patient examination.评估标准化患者考试中重测考生的结构等效性和效标关联效度。

Acad Med. 2011 Oct;86(10):1253-9. doi: 10.1097/ACM.0b013e31822bc0a4.

Characteristics and Implications of Diagnostic Justification Scores Based on the New Patient Note Format of the USMLE Step 2 CS Exam.基于美国医师执照考试第二步临床技能考试新患者记录格式的诊断理由分数的特征及影响

Acad Med. 2015 Nov;90(11 Suppl):S56-62. doi: 10.1097/ACM.0000000000000900.

Correlation between housestaff performance on the United States Medical Licensing Examination and standardized patient encounters.美国医师执照考试中住院医师表现与标准化病人问诊之间的相关性。

Mt Sinai J Med. 2005 Jan;72(1):47-9.

The generalizability of documentation scores from the USMLE Step 2 Clinical Skills examination.美国医师执照考试第二步临床技能考试文档评分的可推广性。

Acad Med. 2008 Oct;83(10 Suppl):S41-4. doi: 10.1097/ACM.0b013e318183cd1d.

The relationship between second-year medical students' OSCE scores and USMLE Step 2 scores.二年级医学生客观结构化临床考试（OSCE）成绩与美国医师执照考试第二步（USMLE Step 2）成绩之间的关系。

J Eval Clin Pract. 2007 Dec;13(6):901-5. doi: 10.1111/j.1365-2753.2006.00768.x.

Validity Evidence and Scoring Guidelines for Standardized Patient Encounters and Patient Notes From a Multisite Study of Clinical Performance Examinations in Seven Medical Schools.来自七所医学院校临床技能考试多中心研究的标准化患者问诊及患者记录的效度证据与评分指南

Acad Med. 2017 Nov;92(11S Association of American Medical Colleges Learn Serve Lead: Proceedings of the 56th Annual Research in Medical Education Sessions):S12-S20. doi: 10.1097/ACM.0000000000001918.