Faculty of ICT, Mahidol University, Nakhon Pathom, Thailand.
Faculty of Dentistry, Thammasat University, Pathum Thani, Thailand.
Comput Methods Programs Biomed. 2018 Jan;153:53-59. doi: 10.1016/j.cmpb.2017.10.001. Epub 2017 Oct 7.
We address the problem of automated outcome assessment in a virtual reality (VR) simulator for endodontic surgery. Outcome assessment is an essential component of any system that provides formative feedback, which requires assessing the outcome, relating it to the procedure, and communicating in a language natural to dental students. This study takes a first step toward automated generation of such comprehensive feedback.
Virtual reference templates are computed based on tooth anatomy and the outcome is assessed with a 3D score cube volume which consists of voxel-level non-linear weighted scores based on the templates. The detailed scores are transformed into standard scoring language used by dental schools. The system was evaluated on fifteen outcome samples that contained optimal results and those with errors including perforation of the walls, floor, and both, as well as various combinations of major and minor over and under drilling errors. Five endodontists who had professional training and varying levels of experiences in root canal treatment participated as raters in the experiment.
Results from evaluation of our system with expert endodontists show a high degree of agreement with expert scores (information based measure of disagreement 0.04-0.21). At the same time they show some disagreement among human expert scores, reflecting the subjective nature of human outcome scoring. The discriminatory power of the AOS scores analyzed with three grade tiers (A, B, C) using the area under the receiver operating characteristic curve (AUC). The AUC values are generally highest for the {AB: C} cutoff which is cutoff at the boundary between clinically acceptable (B) and clinically unacceptable (C) grades.
The objective consistency of computed scores and high degree of agreement with experts make the proposed system a promising addition to existing VR simulators. The translation of detailed level scores into terminology commonly used in dental surgery supports natural communication with students and instructors. With the reference virtual templates created automatically, the approach is robust and is applicable in scoring the outcome of any dental surgery procedure involving the act of drilling.
我们在牙髓治疗手术的虚拟现实（VR）模拟器中解决自动评估结果的问题。结果评估是提供形成性反馈的任何系统的基本组成部分，这需要评估结果，将其与程序相关联，并以牙科学生自然的语言进行交流。本研究朝着自动生成此类综合反馈迈出了第一步。
基于牙齿解剖结构计算虚拟参考模板，通过由模板得出的体素级非线性加权分数的 3D 评分立方体体积来评估结果。详细的分数转换为牙科学校使用的标准评分语言。该系统在包含最佳结果和包含穿孔、地板以及穿孔和地板的组合以及各种大小和小钻误差的十五个结果样本上进行了评估。五位牙髓专家参与了实验，他们具有根管治疗的专业培训和不同水平的经验。
使用专家牙髓专家评估我们的系统的结果表明，与专家评分高度一致（基于信息的不一致度量为 0.04-0.21）。同时，它们也显示出人类专家评分之间存在一些分歧，反映了人类结果评分的主观性。使用接收者操作特征曲线（AUC）下的面积分析 AOS 评分的区分能力。AUC 值通常在使用三个等级（A、B、C）的 {AB: C} 截止值时最高，该截止值位于临床可接受（B）和临床不可接受（C）等级之间的边界处。
计算得分的客观一致性和与专家的高度一致性使所提出的系统成为现有 VR 模拟器的有前途的补充。详细等级得分转换为牙科手术中常用的术语支持与学生和教师进行自然交流。由于参考虚拟模板是自动创建的，因此该方法具有鲁棒性，适用于评估涉及钻孔操作的任何牙科手术程序的结果。
