Department of Orthodontics, Medway Maritime Hospital, Windmill Road, Gillingham, Kent, ME7 5NY, UK.
Department of Orthodontics and Dentofacial Orthopedics, Dental School/Medical Faculty, University of Bern, Freiburgstrasse7 CH-3010, Bern, Switzerland.
Br Dent J. 2022 May;232(10):721-726. doi: 10.1038/s41415-022-4246-2. Epub 2022 May 27.
Aim To investigate reliability of the Easy IOTN app between clinicians with different levels of experience in determining Index of Orthodontic Treatment Need (IOTN) Dental Health Component (DHC) and Aesthetic Component (AC) scores from study models. The accuracy of each clinician in discriminating treatment need using the app against the 'gold standard' conventional assessment at the threshold of treatment acceptance criteria was also explored.Materials and methods In total, 150 sets of pre-treatment study models were assessed by six clinicians using the app on two separate occasions (T1 and T2). A single IOTN-calibrated clinician also scored the models using the conventional technique. Clinician scores for both intra- and inter-rater reliability were assessed using Cohen's Kappa. The performance of each clinician in discriminating treatment need using the app against the conventional assessment method at the threshold of treatment acceptance criteria was also assessed using the area under the curve-receiver operating characteristic.Results The intra-rater agreement for the clinician undertaking the conventional assessment of the models was 1.0. Intra-rater agreement scores for clinicians using the Easy IOTN app ranged between 0.37-0.87 (DHC) and 0.22-0.44 (AC). Inter-rater agreement scores at T2 were 0.59 (DHC) and 0.23 (AC). Based on the IOTN DHC, all clinicians displayed an excellent level of accuracy in determining malocclusions qualifying for treatment (range 81.7-90.0%). Based on the IOTN AC, all clinicians showed an acceptable level of accuracy in determining malocclusions qualifying for treatment (range 71.9-79.2%).Conclusions The Easy IOTN app was shown to have moderate inter-rater reliability. Variation in the intra-rater reliability was evident between clinicians of different grades/level of experience. Importantly, the diagnostic accuracy of the app to discriminate between malocclusions that qualify for NHS treatment was rated as excellent (IOTN DHC) and acceptable (IOTN AC) and independent of clinician grade or level of experience.
研究不同经验水平的临床医生使用 Easy IOTN 应用程序（APP）在确定研究模型的正畸治疗需求指数（IOTN）的口腔健康成分（DHC）和美学成分（AC）评分方面的可靠性。还探讨了每位临床医生使用 APP 进行诊断治疗需求的准确性，以及 APP 对接受治疗标准阈值的传统评估的准确性。
共有 150 套治疗前模型由 6 位临床医生分别使用 APP 进行了两次评估（T1 和 T2）。一位经过 IOTN 校准的临床医生也使用传统技术对模型进行了评分。使用 Cohen Kappa 评估了临床医生对两种评估方法的内部和外部可靠性的评分。还使用曲线下面积-接受者操作特征来评估每位临床医生使用 APP 进行诊断治疗需求的准确性，与传统评估方法在接受治疗标准的阈值方面的比较。
进行模型传统评估的临床医生的内部一致性评分为 1.0。使用 Easy IOTN APP 的临床医生的内部一致性评分范围为 0.37-0.87（DHC）和 0.22-0.44（AC）。T2 时的外部一致性评分为 0.59（DHC）和 0.23（AC）。根据 IOTN DHC，所有临床医生在确定符合治疗标准的错畸形方面都具有极好的准确性（范围为 81.7-90.0%）。根据 IOTN AC，所有临床医生在确定符合治疗标准的错畸形方面都具有可接受的准确性（范围为 71.9-79.2%）。
Easy IOTN APP 显示出中等的外部一致性。不同等级/经验水平的临床医生之间的内部一致性存在差异。重要的是，APP 区分 NHS 治疗所需的错畸形的诊断准确性被评为极好（IOTN DHC）和可接受（IOTN AC），与临床医生的等级或经验水平无关。
