Department of Information Engineering and Mathematics, University of Siena, Via Roma 56, Siena, Italy.
Department of Information Engineering and Mathematics, University of Siena, Via Roma 56, Siena, Italy.
Comput Methods Programs Biomed. 2020 Feb;184:105268. doi: 10.1016/j.cmpb.2019.105268. Epub 2019 Dec 17.
Deep learning models and specifically Convolutional Neural Networks (CNNs) are becoming the leading approach in many computer vision tasks, including medical image analysis. Nevertheless, the CNN training usually requires large sets of supervised data, which are often difficult and expensive to obtain in the medical field. To address the lack of annotated images, image generation is a promising method, which is becoming increasingly popular in the computer vision community. In this paper, we present a new approach to the semantic segmentation of bacterial colonies in agar plate images, based on deep learning and synthetic image generation, to increase the training set size. Indeed, semantic segmentation of bacterial colony is the basis for infection recognition and bacterial counting in Petri plate analysis.
A convolutional neural network (CNN) is used to separate the bacterial colonies from the background. To face the lack of annotated images, a novel engine is designed - which exploits a generative adversarial network to capture the typical distribution of the bacterial colonies on agar plates - to generate synthetic data. Then, bacterial colony patches are superimposed on existing background images, taking into account both the local appearance of the background and the intrinsic opacity of the bacterial colonies, and a style transfer algorithm is used for further improve visual realism.
The proposed deep learning approach has been tested on the only public dataset available with pixel-level annotations for bacterial colony semantic segmentation in agar plates. The role of including synthetic data in the training of a segmentation CNN has been evaluated, showing how comparable performances can be obtained with respect to the use of real images. Qualitative results are also reported for a second public dataset in which the segmentation annotations are not provided.
The use of a small set of real data, together with synthetic images, allows obtaining comparable results with respect to using a complete set of real images. Therefore, the proposed synthetic data generator is able to address the scarcity of biomedical data and provides a scalable and cheap alternative to human ground-truth supervision.
深度学习模型，特别是卷积神经网络（CNN），在包括医学图像分析在内的许多计算机视觉任务中成为主流方法。然而，CNN 训练通常需要大量的有监督数据，而在医学领域中，这些数据通常难以获取且成本高昂。为了解决注释图像缺乏的问题，图像生成是一种很有前途的方法，在计算机视觉领域越来越受欢迎。在本文中，我们提出了一种新的基于深度学习和合成图像生成的琼脂平板图像中细菌菌落的语义分割方法，以增加训练集的大小。实际上，细菌菌落的语义分割是识别感染和对培养皿分析中细菌进行计数的基础。
使用卷积神经网络（CNN）将细菌从背景中分离出来。为了解决注释图像缺乏的问题，设计了一种新的引擎——利用生成对抗网络来捕捉琼脂平板上细菌菌落的典型分布——生成合成数据。然后，将细菌菌落补丁叠加到现有的背景图像上，同时考虑背景的局部外观和细菌的固有不透明度，并使用样式转换算法进一步提高视觉真实感。
所提出的深度学习方法已经在唯一的公共数据集上进行了测试，该数据集提供了琼脂平板上细菌菌落语义分割的像素级注释。评估了在分割 CNN 训练中包含合成数据的作用，展示了与使用真实图像相比可以获得相当的性能。还报告了第二个公共数据集的定性结果，其中未提供分割注释。
使用一小部分真实数据和合成图像可以获得与使用完整的真实图像相当的结果。因此，所提出的合成数据生成器能够解决生物医学数据的稀缺性，并提供一种可扩展且廉价的替代人工真实监督的方法。
