Appl Opt. 2021 Aug 1;60(22):6682-6694. doi: 10.1364/AO.431076.
Different from conventional microimaging techniques, polarization imaging can generate multiple polarization images in a single perspective by changing the polarization angle. However, how to efficiently fuse the information in these multiple polarization images by a convolutional neural network (CNN) is still a challenging problem. In this paper, we propose a hybrid 3D-2D convolutional neural network called MuellerNet, to classify biological cells with Mueller matrix images (MMIs). The MuellerNet includes a normal stream and a polarimetric stream, in which the first Mueller matrix image is taken as the input of normal stream, and the rest MMIs are stacked to form the input of a polarimetric stream. The normal stream is mainly constructed with a backbone network and, in the polarimetric stream, the attention mechanism is used to adaptively assign weights to different convolutional maps. To improve the network's discrimination, a loss function is introduced to simultaneously optimize parameters of the two streams. Two Mueller matrix image datasets are built, which include four types of breast cancer cells and three types of algal cells, respectively. Experiments are conducted on these two datasets with many well-known and recent networks. Results show that the proposed network efficiently improves the classification accuracy and helps to find discriminative features in MMIs.
与传统的微观成像技术不同，偏振成像可以通过改变偏振角在单个视角中生成多个偏振图像。然而，如何通过卷积神经网络（CNN）有效地融合这些多个偏振图像中的信息仍然是一个具有挑战性的问题。在本文中，我们提出了一种称为 MuellerNet 的混合 3D-2D 卷积神经网络，用于使用 Mueller 矩阵图像（MMI）对生物细胞进行分类。MuellerNet 包括一个常规流和一个偏振流，其中第一 Mueller 矩阵图像作为常规流的输入，其余 MMI 堆叠在一起形成偏振流的输入。常规流主要由骨干网络构建，在偏振流中，注意力机制用于自适应地为不同的卷积图分配权重。为了提高网络的辨别能力，引入了一个损失函数来同时优化两个流的参数。构建了两个 Mueller 矩阵图像数据集，分别包含四种乳腺癌细胞和三种藻类细胞。在这两个数据集上进行了许多知名和最新网络的实验。结果表明，所提出的网络有效地提高了分类准确性，并有助于在 MMI 中找到有区别的特征。
