Eindhoven University of Technology, Groene Loper 3, 5612 AE Eindhoven, The Netherlands.
Amsterdam University Medical Center, Meibergdreef 9, 1105 AZ Amsterdam, The Netherlands.
Sensors (Basel). 2020 Jul 24;20(15):4133. doi: 10.3390/s20154133.
Early Barrett's neoplasia are often missed due to subtle visual features and inexperience of the non-expert endoscopist with such lesions. While promising results have been reported on the automated detection of this type of early cancer in still endoscopic images, video-based detection using the temporal domain is still open. The temporally stable nature of video data in endoscopic examinations enables to develop a framework that can diagnose the imaged tissue class over time, thereby yielding a more robust and improved model for spatial predictions. We show that the introduction of Recurrent Neural Network nodes offers a more stable and accurate model for tissue classification, compared to classification on individual images. We have developed a customized Resnet18 feature extractor with four types of classifiers: Fully Connected (FC), Fully Connected with an averaging filter (FC Avg(n = 5)), Long Short Term Memory (LSTM) and a Gated Recurrent Unit (GRU). Experimental results are based on 82 pullback videos of the esophagus with 46 high-grade dysplasia patients. Our results demonstrate that the LSTM classifier outperforms the FC, FC Avg(n = 5) and GRU classifier with an average accuracy of 85.9% compared to 82.2%, 83.0% and 85.6%, respectively. The benefit of our novel implementation for endoscopic tissue classification is the inclusion of spatio-temporal information for improved and robust decision making, and it is the first step towards full temporal learning of esophageal cancer detection in endoscopic video.
早期 Barrett 肿瘤由于其视觉特征细微且非专业内镜医师对此类病变经验不足，常常被漏诊。虽然在对内镜静止图像中这种早期癌症的自动检测方面已经取得了有前景的结果，但基于视频的时域检测仍未得到广泛应用。内镜检查中视频数据的时间稳定性使得能够开发一个可以随时间诊断所成像组织类别的框架，从而为空间预测提供更稳健和改进的模型。我们表明，与在单个图像上进行分类相比，引入递归神经网络节点可为组织分类提供更稳定和准确的模型。我们已经开发了一个定制的 Resnet18 特征提取器，具有四种分类器：全连接 (FC)、带平均滤波器的全连接 (FC Avg(n = 5))、长短时记忆 (LSTM) 和门控循环单元 (GRU)。实验结果基于 46 名高级别发育不良患者的 82 个食管回拉视频。我们的结果表明，与 FC、FC Avg(n = 5)和 GRU 分类器相比，LSTM 分类器的平均准确率为 85.9%，分别为 82.2%、83.0%和 85.6%。我们用于内镜组织分类的新颖实现的优势在于纳入时空信息以进行改进和稳健决策，这是实现内镜视频中食管癌检测的全时域学习的第一步。
