the American Board of Family Medicine, Lexington, KY 40511, USA.
J Am Board Fam Med. 2011 Mar-Apr;24(2):175-80. doi: 10.3122/jabfm.2011.02.100162.
Certification examinations used by American specialty boards have been the sine qua non for demonstrating the knowledge sufficient for attainment of board certification in the United States for more than 75 years. Some people contend that the examination is predominantly a test of superior test-taking skills rather than of family medicine decision-making ability. In an effort to explore the validity of this assertion, we administered the American Board of Family Medicine (ABFM) Certification to examinees who had demonstrated proficiency in taking standardized tests but had limited medical knowledge.
Four nonphysician experts in the field of measurement and testing were administered one version of the 2009 ABFM certification examination. Scaled scores were calculated for each examinee, and psychometric analyses were performed on the examinees responses to examination items and compared with the performance of physicians who took the same examination.
The minimum passing threshold for the examination was a scaled score of 390, corresponding to 57.7% to 61.0% of questions answered correctly, depending on the version of the examination. The 4 nonphysician examinees performed poorly, with scaled scores that ranged from 20 to 160 (mean, 87.5; SD, 57.4). The number of questions answered correctly ranged from 24.0% to 35.1% (mean, 29.2%; SD, 0.05%). Rasch analyses of the examination items revealed that the nonphysician examinees were more likely to use guessing strategies in an effort to answer questions correctly. Distracter analysis suggest near-complete randomness in the nonphysician responses.
Though all 4 nonphysician examinees performed better than would have been predicted by chance alone, none performed well enough to even fall within 8 SE below the passing thresholds; their performance was far below that of almost all physicians who completed the examination. Given that the nonphysicians relied heavily on the identifying cues in the phrasing of items and the manner in which response options were presented, the results affirm the notion that the ABFM certification examination is not primarily a measure of generic test-taking ability but measures information critical to the estimation of a family physician's knowledge sufficient for certification. Item analysis confirmed that items were well written, provided minimal cueing, and required medical knowledge to answer correctly.
美国专业委员会使用的认证考试是 75 多年来证明获得美国委员会认证所需知识的必要条件。有人认为，该考试主要是对优秀应试技巧的测试，而不是对家庭医学决策能力的测试。为了探讨这一说法的有效性，我们对已经证明在标准化考试中表现出色但医学知识有限的考生进行了美国家庭医学委员会（ABFM）认证考试。
四位非医学领域的测量和测试专家接受了 2009 年 ABFM 认证考试的一个版本。为每位考生计算了标准化分数，并对考生对考试题目的反应进行了心理测量分析，并与参加同一考试的医生的表现进行了比较。
考试的最低通过门槛是 390 分的标准化分数，对应答对 57.7%至 61.0%的题目，具体取决于考试版本。这 4 名非医学考生的表现不佳，标准化分数从 20 到 160 分不等（平均值为 87.5，标准差为 57.4）。答对的题目数量从 24.0%到 35.1%不等（平均值为 29.2%，标准差为 0.05%）。对考试题目的 Rasch 分析显示，非医学考生更倾向于使用猜测策略来正确回答问题。干扰项分析表明，非医学考生的回答几乎完全是随机的。
尽管所有 4 名非医学考生的表现都比仅凭机会预测的要好，但没有一名考生的表现好到足以低于通过标准 8 个标准差；他们的表现远低于几乎所有参加考试的医生。鉴于非医学考生严重依赖于项目措辞中的识别线索以及答案选项的呈现方式，结果证实了 ABFM 认证考试主要不是对通用应试能力的衡量，而是对家庭医生认证所需知识的重要信息的衡量。项目分析证实，这些项目写得很好，提供的提示很少，并且需要医学知识才能正确回答。
