Thomas J. Greany, DDS, is Director of Academic Technology Initiatives and Assistant Clinical Professor, Department of Restorative Dentistry, University of Colorado School of Dental Medicine, as well as Assistant Professor, Modern Human Anatomy Program, Department of Cellular and Developmental Biology, University of Colorado School of Medicine; Ala Yassin, DDS, is an MS student, Graduate Periodontics Department, University of Colorado School of Dental Medicine; and K. Chase Lewis, MS, is a DDS student, University of Colorado School of Dental Medicine.
J Dent Educ. 2019 Nov;83(11):1304-1313. doi: 10.21815/JDE.019.132. Epub 2019 Jul 22.
Visual inspection (VI) of dental student "waxups" by faculty has frequently been challenged as subjective and inconsistent. Traditional grading rubrics fail to precisely assess morphology due to coarse detail and inappropriate application of ratio measurement to ordinal data. The aim of this study was to test the hypothesis that VI would be measurably imprecise and inaccurate and to explore development of a superior digital assessment alternative. In fall 2017, six examiners at one U.S. dental school independently evaluated 81 student waxups of tooth #14 using VI. Grades were awarded using a 20-item rubric, corresponding to discrete morphologic features. After inclusion criteria were met, 67 waxups were subsequently scanned with an intraoral scanner; analyzed using three-dimensional surface comparison software; and digitally compared to scans of the same typodonts containing the original tooth #14. Examiner precision and accuracy were evaluated using digital inspection of morphologic regions in the VI rubric. Within this study's limitations, VI exhibited low inter-examiner precision (ICC 0.332) and accuracy ("correctness"), resulting in potentially low-grade validity. Intra-examiner precision for three examiners was low, based on computation of statistically different mean grades (single tail paired t-test), awarded by the same examiners one week apart. One examiner had to leave the study, for whom paired t-test could not be performed. Independent digital evaluation by two examiners exhibited high inter-examiner precision (ICC 0.866) and optimal accuracy. These results affirm the possibility that digital assessment techniques offer improvements in visualization, consistently valid student evaluation, and optimal self-evaluation and self-correction.
口腔医学生蜡型的肉眼检查（VI）一直以来都受到质疑，因为其主观性和不一致性。传统的评分标准由于细节粗糙且不恰当地将比率测量应用于序数数据，因此无法精确评估形态。本研究旨在检验 VI 可衡量的不精确和不准确的假设，并探讨开发更好的数字化评估替代方案。2017 年秋季，美国一所牙科学校的六名考官分别使用 VI 独立评估了 81 名学生的 14 号牙蜡型。使用 20 项标准对等级进行评定，对应离散的形态特征。在满足纳入标准后，随后对 67 个蜡型进行了口腔内扫描仪扫描; 使用三维表面比较软件进行分析; 并与同一模型中包含原始 14 号牙的扫描仪扫描进行数字化比较。通过 VI 评分标准中形态区域的数字化检查来评估考官的精度和准确性。在本研究的限制范围内，VI 显示出较低的考官间精度（ICC 0.332）和准确性（“正确性”），从而导致潜在的低等级有效性。基于同一考官在一周内授予的统计上不同的平均分数（单侧配对 t 检验），三名考官中的三位考官的内部考官精度较低。一名考官不得不退出研究，对于该考官，无法进行配对 t 检验。两名考官的独立数字化评估显示出较高的考官间精度（ICC 0.866）和最佳准确性。这些结果证实了数字化评估技术在可视化、一致有效的学生评估以及最佳自我评估和自我纠正方面的改进的可能性。
