IEEE Trans Vis Comput Graph. 2023 Mar;29(3):1625-1637. doi: 10.1109/TVCG.2021.3127132. Epub 2023 Jan 30.
Recent advances in high-resolution microscopy have allowed scientists to better understand the underlying brain connectivity. However, due to the limitation that biological specimens can only be imaged at a single timepoint, studying changes to neural projections over time is limited to observations gathered using population analysis. In this article, we introduce NeuRegenerate, a novel end-to-end framework for the prediction and visualization of changes in neural fiber morphology within a subject across specified age-timepoints. To predict projections, we present neuReGANerator, a deep-learning network based on cycle-consistent generative adversarial network (GAN) that translates features of neuronal structures across age-timepoints for large brain microscopy volumes. We improve the reconstruction quality of the predicted neuronal structures by implementing a density multiplier and a new loss function, called the hallucination loss. Moreover, to alleviate artifacts that occur due to tiling of large input volumes, we introduce a spatial-consistency module in the training pipeline of neuReGANerator. Finally, to visualize the change in projections, predicted using neuReGANerator, NeuRegenerate offers two modes: (i) neuroCompare to simultaneously visualize the difference in the structures of the neuronal projections, from two age domains (using structural view and bounded view), and (ii) neuroMorph, a vesselness-based morphing technique to interactively visualize the transformation of the structures from one age-timepoint to the other. Our framework is designed specifically for volumes acquired using wide-field microscopy. We demonstrate our framework by visualizing the structural changes within the cholinergic system of the mouse brain between a young and old specimen.
近年来,高分辨率显微镜技术的进步使得科学家能够更好地理解大脑连接的基础。然而,由于生物样本只能在单个时间点进行成像的限制,因此只能通过群体分析来研究神经投射随时间的变化。在本文中,我们引入了 NeuRegenerate,这是一种新颖的端到端框架,用于预测和可视化特定年龄时间点内主体内部神经纤维形态的变化。为了预测投射,我们提出了 neuReGANerator,这是一种基于循环一致生成对抗网络(GAN)的深度学习网络,它可以在大的脑显微镜体积中跨年龄时间点转换神经元结构的特征。我们通过实现密度乘法器和一种新的损失函数(称为幻觉损失)来提高预测神经元结构的重建质量。此外,为了减轻由于大输入体积的平铺而产生的伪影,我们在 neuReGANerator 的训练管道中引入了一个空间一致性模块。最后,为了可视化使用 neuReGANerator 预测的投射变化,NeuRegenerate 提供了两种模式:(i)neuroCompare,用于同时可视化来自两个年龄域的神经元投射结构的差异(使用结构视图和有界视图),以及(ii)neuroMorph,一种基于血管的变形技术,用于交互式地可视化结构从一个年龄时间点到另一个年龄时间点的变化。我们的框架是专门为使用宽场显微镜获得的体积设计的。我们通过可视化年轻和年老样本之间小鼠大脑中的胆碱能系统内的结构变化来展示我们的框架。