National Board of Osteopathic Medical Examiners, Chicago, IL, USA.
National Board of Osteopathic Medical Examiners, Philadelphia, PA, USA.
J Osteopath Med. 2021 May 13;121(8):687-691. doi: 10.1515/jom-2021-0007.
The Comprehensive Osteopathic Medical Licensing Examination of the United States of America (COMLEX-USA) is a three level examination used as a pathway to licensure for students in osteopathic medical education programs. COMLEX-USA Level 2 includes a written assessment of Fundamental Clinical Sciences for Osteopathic Medical Practice (Level 2-Cognitive Evaluation [L2-CE]) delivered in a computer based format and separate performance evaluation (Level 2-Performance Evaluation [L2-PE]) administered through live encounters with standardized patients. L2-PE was designed to augment L2-CE. It is expected that the two examinations measure related yet distinct constructs.
To explore the concurrent validity of L2-CE with L2-PE.
First attempt test scores were obtained from the National Board of Osteopathic Medical Examiners database for 6,639 candidates who took L2-CE between June 2019 and May 2020 and matched to the students' L2-PE scores. The sample represented all colleges of osteopathic medicine and 97.5% of candidates who took L2-CE during the complete 2019-2020 test cycle. We calculated disattenuated correlations between the total score for L2-CE, the L2-CE scores for the seven competency domains (CD1 through CD7), and the L2-PE scores for the Humanistic Domain (HM) and Biomedical/Biomechanical Domain (BM). All scores were on continuous scales.
Pearson correlations ranged from 0.10 to 0.88 and were all statically significant (p<0.01). L2-CE total score was most strongly correlated with CD2 (0.88) and CD3 (0.85). Pearson correlations between the L2-CE competency domain subscores ranged from 0.17 to 0.70, and correlations which included either HM or BM ranged from 0.10 to 0.34 with the strongest of those correlations being between BM and L2-CE total score (0.34) as well as between HM and BM (0.28).The largest increase between corresponding Pearson and disattenuated correlations was for pairs of scores with lower reliabilities such as CD5 and CD6, which had a Pearson correlation of 0.17 and a disattenuated correlation of 0.68. The smallest increase in correlations was observed in pairs of scores with larger reliabilities such as L2-CE total score and HM, which had a Pearson correlation of 0.23 and a disattenuated correlation of 0.28. The reliability of L2-CE was 0.87, 0.81 for HM, and 0.73 for BM. The reliabilities for the L2-CE competency domain scores ranged from 0.22 to 0.74. The small to moderate correlations between the L2-CE total score and the two L2-PE support the expectation that these examinations measure related but distinct constructs. The correlations between L2-PE and L2-CE competency domain subscores reflect the distribution of items defined by the L2-PE blueprint, providing evidence that the examinations are performing as designed.
This study provides evidence supporting the validity of the blueprints for constructing COMLEX-USA Levels 2-CE and 2-PE examinations in concert with the purpose and nature of the examinations.
美国整骨医学许可考试（COMLEX-USA）是一种三级考试，是骨医学教育项目学生获得行医执照的途径之一。COMLEX-USA 第 2 级包括基础临床科学的书面评估，用于骨医学实践（第 2 级认知评估[L2-CE]），以计算机为基础的格式提供，以及通过与标准化患者进行现场接触进行的单独绩效评估（第 2 级绩效评估[L2-PE]）。L2-PE 的设计目的是增强 L2-CE。预计这两项考试将测量相关但不同的结构。
探讨 L2-CE 与 L2-PE 的并行效度。
从全国骨医学考试委员会数据库中获得了 6639 名考生的首次尝试考试成绩，这些考生于 2019 年 6 月至 2020 年 5 月参加了 L2-CE，并与学生的 L2-PE 分数相匹配。该样本代表了所有的骨医学院和 97.5%的在整个 2019-2020 考试周期中参加 L2-CE 的考生。我们计算了 L2-CE 总分、七个能力领域（CD1 至 CD7）的 L2-CE 分数与人文领域（HM）和生物医学/生物力学领域（BM）的 L2-PE 分数之间的去衰减相关系数。所有分数均为连续尺度。
皮尔逊相关系数范围为 0.10 至 0.88，均具有统计学意义（p<0.01）。L2-CE 总分与 CD2（0.88）和 CD3（0.85）相关性最强。L2-CE 能力领域分项分数之间的皮尔逊相关系数范围为 0.17 至 0.70，包括 HM 或 BM 的相关系数范围为 0.10 至 0.34，其中最强的相关性是 BM 与 L2-CE 总分（0.34）以及 HM 与 BM（0.28）之间的相关性。相应的皮尔逊和去衰减相关系数之间最大的增加是对应分数可靠性较低的情况，例如 CD5 和 CD6，它们的皮尔逊相关系数为 0.17，去衰减相关系数为 0.68。相关性的增加最小的情况是分数可靠性较大的情况，例如 L2-CE 总分和 HM，它们的皮尔逊相关系数为 0.23，去衰减相关系数为 0.28。L2-CE 的可靠性为 0.87，HM 的可靠性为 0.81，BM 的可靠性为 0.73。L2-CE 能力领域分数的可靠性范围为 0.22 至 0.74。L2-CE 总分与两个 L2-PE 之间的小到中度相关性支持这样一种期望，即这些考试测量相关但不同的结构。L2-PE 与 L2-CE 能力领域分项分数之间的相关性反映了 L2-PE 蓝图定义的项目分布，这为考试按设计进行提供了证据。
本研究为 COMLEX-USA 第 2 级 CE 和 2 级 PE 考试蓝图的构建提供了支持，与考试的目的和性质一致。
