Finlay Karen, Norman Geoff R, Keane David R, Stolberg Harald
Diagnostic Radiology, McMaster University, Hamilton, ON.
Can Assoc Radiol J. 2006 Apr;57(2):106-16.
To develop an objective, Web-based tool for evaluating residents' knowledge of diagnostic radiology.
We developed and tested a Web-based evaluation tool (the Diagnostic Radiology Skills Test) that consists of 3 tests, one in each of 3 domains of diagnostic radiology: chest, gastrointestinal, and musculoskeletal imaging. Each test comprises 30 cases representing a range of difficulty in the domain, including normal states, normal variants, typical cases of common diagnoses, and cases with more subtle findings. Cases are presented with a long menu of domain-specific possible diagnoses (response options), each coded for diagnostic appropriateness. Our subjects were 21 residents in postgraduate year (PGY) 2 to 5 and 11 experts in diagnostic radiology. Subjects accessed the tool via a Web site on our Web server. Residents' test results were compared for reliability and validity across domain, case, and training level. In addition, results were correlated with commonly used established and objective evaluation tools.
The tool demonstrated consistent monotonic improvement in performance with training level. It showed acceptable reliability in discriminating between residents at different performance levels, both within and across training levels (r= 0.53 within level and 0.69 across levels). Test results also had concurrent validity against the American College of Radiology In-Training Examination, a widely accepted objective assessment tool (r = 0.65, P < 0.01), and 2 Objective Structured Clinical Examinations (OSCEs) focusing on diagnostic skills (r= 0.78 and r= 0.69, P < 0.01, respectively).
Our study demonstrates the feasibility of a Web-based, standardized, objective assessment method for evaluating residents' performance.
开发一种基于网络的客观工具，用于评估住院医师的诊断放射学知识。
我们开发并测试了一种基于网络的评估工具（诊断放射学技能测试），该工具由3个测试组成，分别针对诊断放射学的3个领域：胸部、胃肠道和肌肉骨骼成像。每个测试包含30个病例，代表该领域一系列难度，包括正常状态、正常变异、常见诊断的典型病例以及具有更细微表现的病例。病例呈现一系列特定领域的可能诊断（应答选项），每个选项都根据诊断适宜性进行编码。我们的研究对象包括21名2至5年级的住院医师和11名诊断放射学专家。研究对象通过我们网络服务器上的网站访问该工具。比较了住院医师在不同领域、病例和培训水平下测试结果的可靠性和有效性。此外，还将结果与常用的既定客观评估工具进行了相关性分析。
该工具显示随着培训水平的提高，性能持续单调改善。在区分不同表现水平的住院医师时，无论在同一培训水平内还是不同培训水平之间，该工具都显示出可接受的可靠性（同一水平内r = 0.53，不同水平间r = 0.69）。测试结果与美国放射学会住院医师培训考试（一种广泛接受的客观评估工具）也具有同时效度（r = 0.65，P < 0.01），以及与两项侧重于诊断技能的客观结构化临床考试（OSCE）具有同时效度（分别为r = 0.78和r = 0.69，P < 0.01）。
我们的研究证明了基于网络的标准化客观评估方法用于评估住院医师表现的可行性。
