Hendricson William D, Rugh John D, Hatch John P, Stark Debra L, Deahl Thomas, Wallmann Elizabeth R
Educational and Faculty Development, Dental School, University of Texas Health Science Center at San Antonio, 7703 Floyd Curl Drive, San Antonio, TX 78229, USA.
J Dent Educ. 2011 Feb;75(2):131-44.
This article reports the validation of an assessment instrument designed to measure the outcomes of training in evidence-based practice (EBP) in the context of dentistry. Four EBP dimensions are measured by this instrument: 1) understanding of EBP concepts, 2) attitudes about EBP, 3) evidence-accessing methods, and 4) confidence in critical appraisal. The instrument-the Knowledge, Attitudes, Access, and Confidence Evaluation (KACE)-has four scales, with a total of thirty-five items: EBP knowledge (ten items), EBP attitudes (ten), accessing evidence (nine), and confidence (six). Four elements of validity were assessed: consistency of items within the KACE scales (extent to which items within a scale measure the same dimension), discrimination (capacity to detect differences between individuals with different training or experience), responsiveness (capacity to detect the effects of education on trainees), and test-retest reliability. Internal consistency of scales was assessed by analyzing responses of second-year dental students, dental residents, and dental faculty members using Cronbach coefficient alpha, a statistical measure of reliability. Discriminative validity was assessed by comparing KACE scores for the three groups. Responsiveness was assessed by comparing pre- and post-training responses for dental students and residents. To measure test-retest reliability, the full KACE was completed twice by a class of freshman dental students seventeen days apart, and the knowledge scale was completed twice by sixteen faculty members fourteen days apart. Item-to-scale consistency ranged from 0.21 to 0.78 for knowledge, 0.57 to 0.83 for attitude, 0.70 to 0.84 for accessing evidence, and 0.87 to 0.94 for confidence. For discrimination, ANOVA and post hoc testing by the Tukey-Kramer method revealed significant score differences among students, residents, and faculty members consistent with education and experience levels. For responsiveness to training, dental students and residents demonstrated statistically significant changes, in desired directions, from pre- to post-test. For the student test-retest, Pearson correlations for KACE scales were as follows: knowledge 0.66, attitudes 0.66, accessing evidence 0.74, and confidence 0.76. For the knowledge scale test-retest by faculty members, the Pearson correlation was 0.79. The construct validity of the KACE is equivalent to that of instruments that assess similar EBP dimensions in medicine. Item consistency for the knowledge scale was more variable than for other KACE scales, a finding also reported for medically oriented EBP instruments. We conclude that the KACE has good discriminative validity, responsiveness to training effects, and test-retest reliability.
本文报告了一种评估工具的验证情况，该工具旨在衡量牙科领域基于循证实践（EBP）培训的成果。该工具测量四个EBP维度：1）对EBP概念的理解，2）对EBP的态度，3）证据获取方法，4）批判性评价的信心。该工具——知识、态度、获取和信心评估（KACE）——有四个量表，共35个项目：EBP知识（10项）、EBP态度（10项）、证据获取（9项）和信心（6项）。评估了效度的四个要素：KACE量表内项目的一致性（量表内项目测量同一维度的程度）、区分度（检测不同培训或经验个体之间差异的能力）、反应度（检测教育对受训者影响的能力）和重测信度。通过使用Cronbach系数α（一种可靠性的统计指标）分析二年级牙科学生、牙科住院医师和牙科教员的回答来评估量表的内部一致性。通过比较三组的KACE分数来评估区分效度。通过比较牙科学生和住院医师培训前后的回答来评估反应度。为了测量重测信度，一年级牙科学生班级相隔17天完成两次完整的KACE，16名教员相隔14天完成两次知识量表。知识量表的项目与量表一致性范围为0.21至0.78，态度为0.57至0.83，证据获取为0.70至0.84，信心为0.87至0.94。对于区分度，方差分析和Tukey-Kramer方法的事后检验显示，学生、住院医师和教员之间的分数差异与教育和经验水平一致。对于培训的反应度，牙科学生和住院医师在测试前到测试后在期望的方向上表现出统计学上的显著变化。对于学生重测，KACE量表的Pearson相关性如下：知识0.66，态度0.66，证据获取0.74，信心0.76。对于教员的知识量表重测，Pearson相关性为0.79。KACE的结构效度与评估医学中类似EBP维度的工具相当。知识量表的项目一致性比其他KACE量表更具变异性，这一发现也在以医学为导向的EBP工具中有所报道。我们得出结论，KACE具有良好的区分效度、对培训效果的反应度和重测信度。
