Department of Internal Medicine, Hallym University College of Medicine, Chuncheon, Republic of Korea.
Institute for Liver and Digestive Diseases, Hallym University, Chuncheon, Republic of Korea.
J Med Internet Res. 2021 Apr 15;23(4):e25167. doi: 10.2196/25167.
In a previous study, we examined the use of deep learning models to classify the invasion depth (mucosa-confined versus submucosa-invaded) of gastric neoplasms using endoscopic images. The external test accuracy reached 77.3%. However, model establishment is labor intense, requiring high performance. Automated deep learning (AutoDL) models, which enable fast searching of optimal neural architectures and hyperparameters without complex coding, have been developed.
The objective of this study was to establish AutoDL models to classify the invasion depth of gastric neoplasms. Additionally, endoscopist-artificial intelligence interactions were explored.
The same 2899 endoscopic images that were employed to establish the previous model were used. A prospective multicenter validation using 206 and 1597 novel images was conducted. The primary outcome was external test accuracy. Neuro-T, Create ML Image Classifier, and AutoML Vision were used in establishing the models. Three doctors with different levels of endoscopy expertise were asked to classify the invasion depth of gastric neoplasms for each image without AutoDL support, with faulty AutoDL support, and with best performance AutoDL support in sequence.
The Neuro-T-based model reached 89.3% (95% CI 85.1%-93.5%) external test accuracy. For the model establishment time, Create ML Image Classifier showed the fastest time of 13 minutes while reaching 82.0% (95% CI 76.8%-87.2%) external test accuracy. While the expert endoscopist's decisions were not influenced by AutoDL, the faulty AutoDL misled the endoscopy trainee and the general physician. However, this was corrected by the support of the best performance AutoDL model. The trainee gained the most benefit from the AutoDL support.
AutoDL is deemed useful for the on-site establishment of customized deep learning models. An inexperienced endoscopist with at least a certain level of expertise can benefit from AutoDL support.
在之前的一项研究中，我们使用内镜图像检查了深度学习模型对胃肿瘤浸润深度（黏膜内局限型与黏膜下浸润型）的分类能力。外部测试准确率达到 77.3%。然而，模型建立过程需要大量的人力，并且需要较高的性能。因此，已经开发出了自动化深度学习（AutoDL）模型，它可以在无需复杂编码的情况下快速搜索最佳的神经结构和超参数。
本研究旨在建立 AutoDL 模型来分类胃肿瘤的浸润深度，并探索内镜医生与人工智能的交互作用。
使用了相同的 2899 张内镜图像来建立之前的模型，并进行了前瞻性多中心验证，使用了 206 张新图像和 1597 张新图像。主要结局指标为外部测试准确率。使用了 Neuro-T、Create ML Image Classifier 和 AutoML Vision 来建立模型。三位具有不同内镜专业知识水平的医生被要求在没有 AutoDL 支持、有错误的 AutoDL 支持和最佳性能的 AutoDL 支持的情况下，依次对每张图像的胃肿瘤浸润深度进行分类。
基于 Neuro-T 的模型达到了 89.3%（95%CI 85.1%-93.5%）的外部测试准确率。在模型建立时间方面，Create ML Image Classifier 用时最短，仅为 13 分钟，达到了 82.0%（95%CI 76.8%-87.2%）的外部测试准确率。虽然专家内镜医生的决策不受 AutoDL 的影响，但错误的 AutoDL 误导了内镜培训生和一般医生。然而，这一点通过最佳性能的 AutoDL 模型的支持得到了纠正。培训生从 AutoDL 的支持中获益最多。
AutoDL 被认为对现场定制深度学习模型的建立有用。具有一定专业知识水平的经验不足的内镜医生可以从 AutoDL 的支持中受益。
