Jones Lyell K, Dimberg Elliot L, Boes Christopher J, Eggers Scott D Z, Dodick David W, Cutsforth-Gregory Jeremy K, Leep Hunderfund Andrea N, Capobianco David J
From the Department of Neurology (L.K.J., C.J.B., S.D.Z.E., J.K.C.-G., A.N.L.H.), Mayo Clinic, Rochester, MN; the Department of Neurology (E.L.D., D.J.C.), Mayo Clinic, Jacksonville, FL; and the Department of Neurology (D.W.D.), Mayo Clinic, Scottsdale, AZ.
Neurology. 2015 Jun 2;84(22):2279-83. doi: 10.1212/WNL.0000000000001641. Epub 2015 May 1.
Beginning in 2014, US neurology residency programs were required to report each trainee's educational progression within 29 neurology Milestone competency domains. Trainee assessment systems will need to be adapted to inform these requirements. The primary aims of this study were to validate neurology resident assessment content using observable practice activities (OPAs) and to develop assessment formats easily translated to the Neurology Milestones.
A modified Delphi technique was used to establish consensus perceptions of importance of 73 neurology OPAs among neurology educators and trainees at 3 neurology residency programs. A content validity score (CVS) was derived for each neurology OPA, with scores ≥4.0 determined in advance to indicate sufficient content validity.
The mean CVS for all OPAs was 4.4 (range 3.5-5.0). Fifty-seven (78%) OPAs had a CVS ≥4.0, leaving 16 (22%) below the pre-established threshold for content validity. Trainees assigned a higher importance to individual OPAs (mean CVS 4.6) compared to faculty (mean 4.4, p = 0.016), but the effect size was small (η(2) = 0.10). There was no demonstrated effect of length of education experience on perceived importance of neurology OPAs (p = 0.23). Two sample resident assessment formats were developed, one using neurology OPAs alone and another using a combination of neurology OPAs and the Neurology Milestones.
This study provides neurology training programs with content validity evidence for items to include in resident assessments, and sample assessment formats that directly translate to the Neurology Milestones. Length of education experience has little effect on perceptions of neurology OPA importance.
自2014年起，美国神经病学住院医师培训项目被要求报告每位学员在29个神经病学里程碑能力领域内的教育进展情况。学员评估系统需要进行调整以满足这些要求。本研究的主要目的是使用可观察实践活动（OPA）验证神经病学住院医师评估内容，并开发易于转化为神经病学里程碑的评估形式。
采用改良的德尔菲技术，在3个神经病学住院医师培训项目的神经病学教育工作者和学员中，就73项神经病学OPA的重要性达成共识性看法。为每个神经病学OPA得出一个内容效度分数（CVS），预先确定分数≥4.0表示有足够的内容效度。
所有OPA的平均CVS为4.4（范围3.5 - 5.0）。57项（78%）OPA的CVS≥4.0，16项（22%）低于预先设定的内容效度阈值。与教员相比，学员对单个OPA赋予了更高的重要性（平均CVS 4.6）（教员平均为4.4，p = 0.016），但效应量较小（η(2)=0.10）。教育经历的长短对神经病学OPA重要性的认知没有显著影响（p = 0.23）。开发了两种住院医师评估形式样本，一种仅使用神经病学OPA，另一种使用神经病学OPA与神经病学里程碑的组合。
本研究为神经病学培训项目提供了住院医师评估中应包含项目的内容效度证据，以及可直接转化为神经病学里程碑的样本评估形式。教育经历的长短对神经病学OPA重要性的认知影响很小。
