Li Chenghuan, Xiao Meihua, Li Zehuan, Chen Fangping, Wang Dingli
Software of School, East China JiaoTong University, Nanchang, JiangXi, China.
Jiangxi University of Software Professional Technology, Nanchang, JiangXi, China.
PeerJ Comput Sci. 2024 Oct 22;10:e2403. doi: 10.7717/peerj-cs.2403. eCollection 2024.
Constructing global context information and local fine-grained information simultaneously is extremely important for single-view 3D reconstruction. In this study, we propose a network that uses spatial dimension attention and channel dimension attention for single-view 3D reconstruction, named R3Davit. Specifically, R3Davit consists of an encoder and a decoder, where the encoder comes from the Davit backbone network. Different from the previous transformer backbone network, Davit focuses on spatial and channel dimensions, fully constructing global context information and local fine-grained information while maintaining linear complexity. To effectively learn features from dual attention and maintain the overall inference speed of the network, we do not use a self-attention layer in the decoder but design a decoder with a nonlinear reinforcement block, a selective state space model block, and an up-sampling Residual Block. The nonlinear enhancement block is used to enhance the nonlinear learning ability of the network. The Selective State Space Model Block replaces the role of the self-attention layer and maintains linear complexity. The up-sampling Residual Block converts voxel features into a voxel model while retaining the voxels of this layer. Features are used in the up-sampling block of the next layer. Experiments on the synthetic dataset ShapeNet and ShapeNetChairRFC with random background show that our method outperforms recent state of the art (SOTA) methods, we lead by 1% and 2% in IOU and F1 scores, respectively. Simultaneously, experiments on the real-world dataset Pix3d fully prove the robustness of our method. The code will be available at https://github.com/epicgzs1112/R3Davit.
同时构建全局上下文信息和局部细粒度信息对于单视图三维重建极为重要。在本研究中，我们提出了一种用于单视图三维重建的网络，该网络使用空间维度注意力和通道维度注意力，名为R3Davit。具体而言，R3Davit由一个编码器和一个解码器组成，其中编码器来自Davit骨干网络。与先前的Transformer骨干网络不同，Davit关注空间和通道维度，在保持线性复杂度的同时充分构建全局上下文信息和局部细粒度信息。为了有效地从双重注意力中学习特征并保持网络的整体推理速度，我们在解码器中不使用自注意力层，而是设计了一个具有非线性增强块、选择性状态空间模型块和上采样残差块的解码器。非线性增强块用于增强网络的非线性学习能力。选择性状态空间模型块取代了自注意力层的作用并保持线性复杂度。上采样残差块将体素特征转换为体素模型，同时保留该层的体素。特征在下一层的上采样块中使用。在具有随机背景的合成数据集ShapeNet和ShapeNetChairRFC上进行的实验表明，我们的方法优于最近的最先进（SOTA）方法，我们在交并比（IOU）和F1分数上分别领先1%和2%。同时，在真实世界数据集Pix3d上进行的实验充分证明了我们方法的鲁棒性。代码将在https://github.com/epicgzs1112/R3Davit上提供。
