Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, CA, 94720, USA.
Department of Radiation Oncology, University of California, San Francisco, CA, 94158, USA.
Sci Rep. 2022 May 4;12(1):7229. doi: 10.1038/s41598-022-10886-6.
Millimeter-scale multi-cellular level imagers enable various applications, ranging from intraoperative surgical navigation to implantable sensors. However, the tradeoffs for miniaturization compromise resolution, making extracting 3D cell locations challenging-critical for tumor margin assessment and therapy monitoring. This work presents three machine-learning-based modules that extract spatial information from single image acquisitions using custom-made millimeter-scale imagers. The neural networks were trained on synthetically-generated (using Perlin noise) cell images. The first network is a convolutional neural network estimating the depth of a single layer of cells, the second is a deblurring module correcting for the point spread function (PSF). The final module extracts spatial information from a single image acquisition of a 3D specimen and reconstructs cross-sections, by providing a layered "map" of cell locations. The maximum depth error of the first module is 100 µm, with 87% test accuracy. The second module's PSF correction achieves a least-square-error of only 4%. The third module generates a binary "cell" or "no cell" per-pixel labeling with an accuracy ranging from 89% to 85%. This work demonstrates the synergy between ultra-small silicon-based imagers that enable in vivo imaging but face a trade-off in spatial resolution, and the processing power of neural networks to achieve enhancements beyond conventional linear optimization techniques.
毫米级多细胞水平成像仪可实现各种应用，从术中手术导航到可植入传感器。然而，微型化的权衡会影响分辨率，使得提取 3D 细胞位置变得具有挑战性，这对于肿瘤边缘评估和治疗监测至关重要。本工作提出了三个基于机器学习的模块，这些模块使用定制的毫米级成像仪从单个图像采集提取空间信息。神经网络是在使用 Perlin 噪声生成的合成细胞图像上进行训练的。第一个网络是一个卷积神经网络，用于估计单层细胞的深度，第二个是一个去模糊模块，用于校正点扩散函数（PSF）。最后一个模块从 3D 标本的单个图像采集提取空间信息，并通过提供细胞位置的分层“地图”来重建横截面。第一个模块的最大深度误差为 100 µm，测试准确率为 87%。第二个模块的 PSF 校正达到了仅 4%的最小二乘误差。第三个模块对每个像素进行二进制“细胞”或“无细胞”标记，准确率在 89%到 85%之间。本工作展示了超小硅基成像仪与神经网络处理能力之间的协同作用，前者可以实现体内成像，但在空间分辨率方面存在权衡，后者可以实现超越传统线性优化技术的增强。
