Columbia University Medical Center, Harkness Eye Institute, New York, NY.
Visulytix Ltd.
J Glaucoma. 2019 Dec;28(12):1029-1034. doi: 10.1097/IJG.0000000000001319.
Pegasus outperformed 5 of the 6 ophthalmologists in terms of diagnostic performance, and there was no statistically significant difference between the deep learning system and the "best case" consensus between the ophthalmologists. The agreement between Pegasus and gold standard was 0.715, whereas the highest ophthalmologist agreement with the gold standard was 0.613. Furthermore, the high sensitivity of Pegasus makes it a valuable tool for screening patients with glaucomatous optic neuropathy.
The purpose of this study was to evaluate the performance of a deep learning system for the identification of glaucomatous optic neuropathy.
Six ophthalmologists and the deep learning system, Pegasus, graded 110 color fundus photographs in this retrospective single-center study. Patient images were randomly sampled from the Singapore Malay Eye Study. Ophthalmologists and Pegasus were compared with each other and to the original clinical diagnosis given by the Singapore Malay Eye Study, which was defined as the gold standard. Pegasus' performance was compared with the "best case" consensus scenario, which was the combination of ophthalmologists whose consensus opinion most closely matched the gold standard. The performance of the ophthalmologists and Pegasus, at the binary classification of nonglaucoma versus glaucoma from fundus photographs, was assessed in terms of sensitivity, specificity and the area under the receiver operating characteristic curve (AUROC), and the intraobserver and interobserver agreements were determined.
Pegasus achieved an AUROC of 92.6% compared with ophthalmologist AUROCs that ranged from 69.6% to 84.9% and the "best case" consensus scenario AUROC of 89.1%. Pegasus had a sensitivity of 83.7% and a specificity of 88.2%, whereas the ophthalmologists' sensitivity ranged from 61.3% to 81.6% and specificity ranged from 80.0% to 94.1%. The agreement between Pegasus and gold standard was 0.715, whereas the highest ophthalmologist agreement with the gold standard was 0.613. Intraobserver agreement ranged from 0.62 to 0.97 for ophthalmologists and was perfect (1.00) for Pegasus. The deep learning system took ∼10% of the time of the ophthalmologists in determining classification.
Pegasus outperformed 5 of the 6 ophthalmologists in terms of diagnostic performance, and there was no statistically significant difference between the deep learning system and the "best case" consensus between the ophthalmologists. The high sensitivity of Pegasus makes it a valuable tool for screening patients with glaucomatous optic neuropathy. Future work will extend this study to a larger sample of patients.
在诊断性能方面，Pegasus 优于 6 位眼科医生中的 5 位，深度学习系统与眼科医生的“最佳情况”共识之间没有统计学上的显著差异。Pegasus 与金标准的一致性为 0.715，而最高的眼科医生与金标准的一致性为 0.613。此外，Pegasus 的高灵敏度使其成为筛查青光眼视神经病变患者的有价值的工具。
本研究旨在评估一种用于识别青光眼视神经病变的深度学习系统的性能。
在这项回顾性单中心研究中，6 位眼科医生和深度学习系统 Pegasus 对 110 张彩色眼底照片进行了分级。患者图像是从新加坡马来眼研究中随机抽取的。将眼科医生和 Pegasus 与彼此以及新加坡马来眼研究给出的原始临床诊断进行比较，后者被定义为金标准。将 Pegasus 的性能与“最佳情况”共识方案进行了比较，“最佳情况”共识方案是指最接近金标准的眼科医生共识意见的组合。从眼底照片的非青光眼与青光眼的二进制分类评估眼科医生和 Pegasus 的性能，包括敏感性、特异性和接收者操作特征曲线 (AUROC) 下的面积，并确定观察者内和观察者间的一致性。
与眼科医生的 AUROC 范围为 69.6%至 84.9%和“最佳情况”共识方案的 AUROC 为 89.1%相比，Pegasus 的 AUROC 为 92.6%。Pegasus 的灵敏度为 83.7%，特异性为 88.2%，而眼科医生的灵敏度范围为 61.3%至 81.6%，特异性范围为 80.0%至 94.1%。Pegasus 与金标准的一致性为 0.715，而最高的眼科医生与金标准的一致性为 0.613。观察者内的一致性范围为眼科医生的 0.62 至 0.97，而 Pegasus 的一致性则为完美（1.00）。深度学习系统在确定分类方面花费的时间约为眼科医生的 10%。
在诊断性能方面，Pegasus 优于 6 位眼科医生中的 5 位，深度学习系统与眼科医生的“最佳情况”共识之间没有统计学上的显著差异。Pegasus 的高灵敏度使其成为筛查青光眼视神经病变患者的有价值的工具。未来的工作将将这项研究扩展到更大的患者样本。
