Xu Pengcheng, Kim Kyungsang, Koh Jeongwan, Wu Dufan, Rim Lee Yu, Young Park Soo, Young Tak Won, Liu Huafeng, Li Quanzheng
College of Optical Science and Engineering, Zhejiang University, Hangzhou, People's Republic of China.
Massachusetts General Hospital and Harvard Medical School, Radiology Department, 55 Fruit Street, Boston, MA 02114, United States of America.
Phys Med Biol. 2021 Nov 26;66(23). doi: 10.1088/1361-6560/ac3935.
Segmentation has been widely used in diagnosis, lesion detection, and surgery planning. Although deep learning (DL)-based segmentation methods currently outperform traditional methods, most DL-based segmentation models are computationally expensive and memory inefficient, which are not suitable for the intervention of liver surgery. To address this issue, a simple solution is to make a segmentation model very small for the fast inference time, however, there is a trade-off between the model size and performance. In this paper, we propose a DL-based real-time 3-D liver CT segmentation method, where knowledge distillation (KD) method, known as knowledge transfer from teacher to student models, is incorporated to compress the model while preserving the performance. Because it is well known that the knowledge transfer is inefficient when the disparity of teacher and student model sizes is large, we propose a growing teacher assistant network (GTAN) to gradually learn the knowledge without extra computational cost, which can efficiently transfer knowledge even with the large gap of teacher and student model sizes. In our results, dice similarity coefficient of the student model with KD improved 1.2% (85.9% to 87.1%) compared to the student model without KD, which is a similar performance of the teacher model using only 8% (100k) parameters. Furthermore, with a student model of 2% (30k) parameters, the proposed model using the GTAN improved the dice coefficient about 2% compared to the student model without KD, and the inference time is 13 ms per a 3-D image. Therefore, the proposed method has a great potential for intervention in liver surgery as well as in many real-time applications.
分割技术已广泛应用于诊断、病变检测和手术规划。尽管基于深度学习（DL）的分割方法目前优于传统方法，但大多数基于DL的分割模型计算成本高且内存效率低，不适用于肝脏手术干预。为了解决这个问题，一个简单的解决方案是使分割模型非常小以实现快速推理时间，然而，模型大小和性能之间存在权衡。在本文中，我们提出了一种基于DL的实时三维肝脏CT分割方法，其中引入了知识蒸馏（KD）方法，即从教师模型到学生模型的知识转移，以在保持性能的同时压缩模型。由于众所周知，当教师模型和学生模型大小差异很大时，知识转移效率低下，我们提出了一种增长型教师辅助网络（GTAN），以在不增加额外计算成本的情况下逐步学习知识，即使教师模型和学生模型大小差距很大，也能有效地转移知识。在我们的结果中，与没有KD的学生模型相比，使用KD的学生模型的骰子相似系数提高了1.2%（从85.9%提高到87.1%），这与仅使用8%（100k）参数的教师模型的性能相似。此外，对于一个具有2%（30k）参数的学生模型，与没有KD的学生模型相比，使用GTAN的所提出模型的骰子系数提高了约2%，并且每幅三维图像的推理时间为13毫秒。因此，所提出的方法在肝脏手术干预以及许多实时应用中具有很大的潜力。
