Division of Surgical Research, Department of Surgery, Larner College of Medicine, University of Vermont, Burlington, Vermont, United States of America.
Division of Ophthalmology, Department of Surgery, Larner College of Medicine, University of Vermont, Burlington, Vermont, United States of America.
PLoS Negl Trop Dis. 2022 Dec 7;16(12):e0010943. doi: 10.1371/journal.pntd.0010943. eCollection 2022 Dec.
Though significant progress in disease elimination has been made over the past decades, trachoma is the leading infectious cause of blindness globally. Further efforts in trachoma elimination are paradoxically being limited by the relative rarity of the disease, which makes clinical training for monitoring surveys difficult. In this work, we evaluate the plausibility of an Artificial Intelligence model to augment or replace human image graders in the evaluation/diagnosis of trachomatous inflammation-follicular (TF).
We utilized a dataset consisting of 2300 images with a 5% positivity rate for TF. We developed classifiers by implementing two state-of-the-art Convolutional Neural Network architectures, ResNet101 and VGG16, and applying a suite of data augmentation/oversampling techniques to the positive images. We then augmented our data set with additional images from independent research groups and evaluated performance.
Models performed well in minimizing the number of false negatives, given the constraint of the low numbers of images in which TF was present. The best performing models achieved a sensitivity of 95% and positive predictive value of 50-70% while reducing the number images requiring skilled grading by 66-75%. Basic oversampling and data augmentation techniques were most successful at improving model performance, while techniques that are grounded in clinical experience, such as highlighting follicles, were less successful.
The developed models perform well and significantly reduce the burden on graders by minimizing the number of false negative identifications. Further improvements in model skill will benefit from data sets with more TF as well as a range in image quality and image capture techniques used. While these models approach/meet the community-accepted standard for skilled field graders (i.e., Cohen's Kappa >0.7), they are insufficient to be deployed independently/clinically at this time; rather, they can be utilized to significantly reduce the burden on skilled image graders.
尽管在过去几十年中，在消除疾病方面取得了重大进展，但沙眼仍是全球致盲的主要传染病原因。进一步消除沙眼的努力受到该疾病相对罕见的限制，这使得监测调查的临床培训变得困难。在这项工作中，我们评估了人工智能模型在评估/诊断沙眼滤泡性炎症（TF）方面增强或替代人工图像分级员的可行性。
我们使用了一个包含 2300 张图像的数据集，其中 TF 的阳性率为 5%。我们通过实现两种最先进的卷积神经网络架构（ResNet101 和 VGG16）并对阳性图像应用一系列数据增强/过采样技术来开发分类器。然后，我们使用来自独立研究小组的其他图像来扩充我们的数据集，并评估性能。
鉴于 TF 存在的图像数量较少，模型在尽可能减少假阴性数量方面表现良好。表现最好的模型在减少需要熟练分级的图像数量的同时，达到了 95%的敏感性和 50-70%的阳性预测值。最成功的模型是基本的过采样和数据增强技术，而基于临床经验的技术，如突出滤泡，则效果较差。
开发的模型表现良好，通过最大限度地减少假阴性的识别数量，大大减轻了分级员的负担。进一步提高模型技能将受益于更多的 TF 数据集以及图像质量和图像采集技术的范围。虽然这些模型接近/满足社区认可的熟练现场分级员的标准（即 Cohen's Kappa>0.7），但目前还不足以独立/临床部署；相反，它们可以大大减轻熟练图像分级员的负担。
