Department of Computational Medicine and Bioinformatics, University of Michigan, Ann Arbor, MI, USA.
Kellogg Eye Center, Department of Ophthalmology and Visual Sciences, University of Michigan, Ann Arbor, MI, USA.
Transl Vis Sci Technol. 2023 Mar 1;12(3):29. doi: 10.1167/tvst.12.3.29.
To develop a class of new metrics for evaluating the performance of intraocular lens power calculation formulas robust to issues that can arise with AI-based methods.
The dataset consists of surgical information and biometry measurements of 6893 eyes of 5016 cataract patients who received Alcon SN60WF lenses at University of Michigan's Kellogg Eye Center. We designed two types of new metrics: the MAEPI (Mean Absolute Error in Prediction of Intraocular Lens [IOL]) and the CIR (Correct IOL Rate) and compared the new metrics with traditional metrics including the mean absolute error (MAE), median absolute error, and standard deviation. We evaluated the new metrics with simulation analysis, machine learning (ML) methods, as well as existing IOL formulas (Barrett Universal II, Haigis, Hoffer Q, Holladay 1, PearlDGS, and SRK/T).
Results of traditional metrics did not accurately reflect the performance of overfitted ML formulas. By contrast, MAEPI and CIR discriminated between accurate and inaccurate formulas. The standard IOL formulas received low MAEPI and high CIR, which were consistent with the results of the traditional metrics.
MAEPI and CIR provide a more accurate reflection of the real-life performance of AI-based IOL formula than traditional metrics. They should be computed in conjunction with conventional metrics when evaluating the performance of new and existing IOL formulas.
The proposed new metrics would help cataract patients avoid the risks caused by inaccurate AI-based formulas, whose true performance cannot be determined by traditional metrics.
开发一类新的指标，用于评估对基于人工智能方法可能出现的问题具有稳健性的人工晶状体屈光力计算公式的性能。
该数据集包含了 5016 名白内障患者的 6893 只眼的手术信息和生物测量数据，这些患者均在密歇根大学凯洛格眼科中心接受了 Alcon SN60WF 人工晶状体。我们设计了两种新型指标：平均预测眼内人工晶状体误差（Mean Absolute Error in Prediction of Intraocular Lens [IOL]）和正确人工晶状体率（Correct IOL Rate），并将新指标与传统指标（平均绝对误差 MAE、中位数绝对误差和标准差）进行了比较。我们通过模拟分析、机器学习（ML）方法以及现有的人工晶状体公式（Barrett Universal II、Haigis、Hoffer Q、Holladay 1、PearlDGS 和 SRK/T）对新指标进行了评估。
传统指标的结果不能准确反映过拟合 ML 公式的性能。相比之下，MAEPI 和 CIR 能够区分准确和不准确的公式。标准人工晶状体公式的 MAEPI 较低，CIR 较高，这与传统指标的结果一致。
MAEPI 和 CIR 比传统指标更能准确反映基于人工智能的人工晶状体公式在现实生活中的性能。在评估新的和现有的人工晶状体公式的性能时，应该将它们与传统指标一起计算。
陈诗雨
