Departments of Medical Education and Emergency Medicine, University of Illinois College of Medicine, Chicago, IL, USA.
Diversity, Equity, and Inclusion, Accreditation Council for Graduate Medical Education, Chicago, IL, USA.
J Gen Intern Med. 2024 May;39(7):1088-1094. doi: 10.1007/s11606-023-08391-1. Epub 2023 Sep 5.
The lack of a standardized language assessment process for medical students and physicians communicating in a non-English language threatens healthcare quality and safety.
To evaluate the validity of a new rating tool, the Physician Oral Language Observation Matrix (POLOM)™, in assessing medical students' oral communication with Spanish-speaking standardized patients (SPs).
POLOM scores were compared to measures of student medical Spanish proficiency to examine convergent validity and to measures of clinical performance to examine concurrent/criterion validity.
Forty-two students at two schools completed SP encounters between January 2021 and April 2022, and POLOM raters scored the videorecorded performances between January and June 2022.
Two approaches to generating POLOM total scores were investigated: rater average and strict consensus. Convergent validity was examined via the POLOM's correlations with (1) the phone-based Clinician Cultural and Linguistic Assessment (CCLA) and (2) the self-rated Interagency Language Roundtable scale for healthcare (ILR-H). Concurrent/criterion validity was examined via correlations with (1) the Comunicación y Habilidades Interpersonales (CAI) scale, (2) a checklist completed by the SP, and (3) a faculty rating of the student's post-encounter clinical note. Pearson's correlations of r ≥ 0.5 and r ≥ 0.2 were considered evidence of convergent validity and concurrent/criterion validity, respectively.
Both rater average and strict consensus POLOM scores were strongly correlated with ILR-H (r = 0.72) and CCLA (r ≥ 0.60), providing evidence of convergent validity. The POLOM was substantially correlated with the CAI (r ≥ 0.29), the SP Checklist (r = 0.32), and the faculty scoring of the student's clinical note (r ≥ 0.24), providing concurrent/criterion validity evidence.
The POLOM has demonstrated evidence of convergent and concurrent/criterion validity as a measure of medical students' Spanish proficiency during SP encounters. Additional research is needed to evaluate how the POLOM can be implemented with resident and practicing physicians, applied to other health professions, and adapted to other languages.
非英语医学专业学生和医生在交流时缺乏标准化的语言评估过程，这威胁到医疗保健的质量和安全。
评估新的评分工具——医生口语语言观察矩阵（POLOM）™在评估医学生与西班牙语标准化患者（SP）口语交流方面的有效性。
将 POLOM 评分与学生医学西班牙语水平的测量结果进行比较，以检验其收敛效度；并与临床绩效的测量结果进行比较，以检验其同时/标准效度。
2021 年 1 月至 2022 年 4 月期间，两所学校的 42 名学生完成了 SP 接触，2022 年 1 月至 6 月，POLOM 评分者对视频记录的表现进行了评分。
研究了两种生成 POLOM 总分的方法：评分者平均值和严格共识。通过 POLOM 与（1）基于电话的临床医生文化和语言评估（CCLA）和（2）自我评估的医疗保健机构间语言圆桌会议量表（ILR-H）的相关性来检验收敛效度。通过与（1）Comunicación y Habilidades Interpersonales（CAI）量表、（2）SP 完成的检查表和（3）教师对学生接触后临床记录的评分的相关性来检验同时/标准效度。Pearson 相关系数 r≥0.5 和 r≥0.2 分别被认为是收敛效度和同时/标准效度的证据。
评分者平均值和严格共识 POLOM 评分与 ILR-H（r=0.72）和 CCLA（r≥0.60）均呈高度相关，提供了收敛效度的证据。POLOM 与 CAI（r≥0.29）、SP 检查表（r=0.32）和教师对学生临床记录的评分（r≥0.24）呈显著相关，提供了同时/标准效度的证据。
POLOM 作为评估医学生在 SP 接触期间西班牙语水平的一种手段，已证明具有收敛和同时/标准效度的证据。需要进一步研究如何将 POLOM 应用于住院医师和执业医师、应用于其他健康专业以及应用于其他语言。
