Australian e-Health Research Centre, Commonwealth Scientific and Industrial Research Organisation, Perth, Western Australia, Australia.
Department of Health Care Policy, Harvard Medical School, Boston, Massachusetts.
JAMA Netw Open. 2018 Sep 7;1(5):e182665. doi: 10.1001/jamanetworkopen.2018.2665.
There has been wide interest in using artificial intelligence (AI)-based grading of retinal images to identify diabetic retinopathy, but such a system has never been deployed and evaluated in clinical practice.
To describe the performance of an AI system for diabetic retinopathy deployed in a primary care practice.
DESIGN, SETTING, AND PARTICIPANTS: Diagnostic study of patients with diabetes seen at a primary care practice with 4 physicians in Western Australia between December 1, 2016, and May 31, 2017. A total of 193 patients consented for the study and had retinal photographs taken of their eyes. Three hundred eighty-six images were evaluated by both the AI-based system and an ophthalmologist.
Sensitivity and specificity of the AI system compared with the gold standard of ophthalmologist evaluation.
Of the 193 patients (93 [48%] female; mean [SD] age, 55 [17] years [range, 18-87 years]), the AI system judged 17 as having diabetic retinopathy of sufficient severity to require referral. The system correctly identified 2 patients with true disease and misclassified 15 as having disease (false-positives). The resulting specificity was 92% (95% CI, 87%-96%), and the positive predictive value was 12% (95% CI, 8%-18%). Many false-positives were driven by inadequate image quality (eg, dirty lens) and sheen reflections.
The results demonstrate both the potential and the challenges of using AI systems to identify diabetic retinopathy in clinical practice. Key challenges include the low incidence rate of disease and the related high false-positive rate as well as poor image quality. Further evaluations of AI systems in primary care are needed.
人们对使用基于人工智能（AI）的视网膜图像分级来识别糖尿病视网膜病变产生了广泛的兴趣，但这种系统从未在临床实践中部署和评估过。
描述在初级保健实践中部署的用于糖尿病视网膜病变的 AI 系统的性能。
设计、设置和参与者：这是一项在澳大利亚西部的一家初级保健诊所进行的患者诊断性研究，该诊所共有 4 名医生。研究时间为 2016 年 12 月 1 日至 2017 年 5 月 31 日，共纳入 193 名同意参与研究并接受眼部视网膜照片拍摄的糖尿病患者。共有 386 张图像由 AI 系统和眼科医生进行评估。
AI 系统与眼科医生评估的金标准相比的敏感性和特异性。
在 193 名患者中（93 名[48%]为女性；平均[SD]年龄为 55[17]岁[范围为 18-87 岁]），AI 系统判断 17 名患者的糖尿病视网膜病变严重程度足以需要转诊。该系统正确识别出 2 名真正患有疾病的患者，并错误地将 15 名患者误诊为患有疾病（假阳性）。因此，特异性为 92%（95%CI，87%-96%），阳性预测值为 12%（95%CI，8%-18%）。许多假阳性是由图像质量差（如镜头脏污）和光泽反射引起的。
结果表明，使用 AI 系统在临床实践中识别糖尿病视网膜病变具有潜在的可能性和挑战。主要挑战包括疾病的低发生率和相关的高假阳性率以及图像质量差。需要进一步在初级保健中评估 AI 系统。
