Yang Xin, Lu Xuqi, Xie Pengyao, Guo Ziyue, Fang Hui, Fu Haowei, Hu Xiaochun, Sun Zhenbiao, Cen Haiyan
College of Biosystems Engineering and Food Science, Zhejiang University, Hangzhou 310058, China.
Key Laboratory of Spectroscopy Sensing, Ministry of Agriculture and Rural Affairs, Hangzhou 310058, China.
Plant Phenomics. 2024 Dec 5;6:0279. doi: 10.34133/plantphenomics.0279. eCollection 2024.
The rice panicle traits substantially influence grain yield, making them a primary target for rice phenotyping studies. However, most existing techniques are limited to controlled indoor environments and have difficulty in capturing the rice panicle traits under natural growth conditions. Here, we developed PanicleNeRF, a novel method that enables high-precision and low-cost reconstruction of rice panicle three-dimensional (3D) models in the field based on the video acquired by the smartphone. The proposed method combined the large model Segment Anything Model (SAM) and the small model You Only Look Once version 8 (YOLOv8) to achieve high-precision segmentation of rice panicle images. The neural radiance fields (NeRF) technique was then employed for 3D reconstruction using the images with 2D segmentation. Finally, the resulting point clouds are processed to successfully extract panicle traits. The results show that PanicleNeRF effectively addressed the 2D image segmentation task, achieving a mean F1 score of 86.9% and a mean Intersection over Union (IoU) of 79.8%, with nearly double the boundary overlap (BO) performance compared to YOLOv8. As for point cloud quality, PanicleNeRF significantly outperformed traditional SfM-MVS (structure-from-motion and multi-view stereo) methods, such as COLMAP and Metashape. The panicle length was then accurately extracted with the rRMSE of 2.94% for and 1.75% for rice. The panicle volume estimated from 3D point clouds strongly correlated with the grain number ( = 0.85 for and 0.82 for ) and grain mass (0.80 for and 0.76 for ). This method provides a low-cost solution for high-throughput in-field phenotyping of rice panicles, accelerating the efficiency of rice breeding.
水稻穗部性状对粮食产量有重大影响，使其成为水稻表型研究的主要目标。然而，大多数现有技术仅限于可控的室内环境，难以捕捉自然生长条件下的水稻穗部性状。在此，我们开发了PanicleNeRF，这是一种基于智能手机采集的视频在田间实现高精度、低成本重建水稻穗三维（3D）模型的新方法。该方法结合了大型模型“分割一切模型”（SAM）和小型模型“你只看一次”第8版（YOLOv8），以实现水稻穗图像的高精度分割。然后利用神经辐射场（NeRF）技术，对具有二维分割的图像进行三维重建。最后，对生成的点云进行处理，成功提取穗部性状。结果表明，PanicleNeRF有效地解决了二维图像分割任务，平均F1分数达到86.9%，平均交并比（IoU）为79.8%，边界重叠（BO）性能几乎是YOLOv8的两倍。在点云质量方面，PanicleNeRF明显优于传统的结构光运动和多视图立体（SfM-MVS）方法，如COLMAP和Metashape。然后准确提取穗长，粳稻的相对均方根误差（rRMSE）为2.94%，籼稻为1.75%。从三维点云估计的穗体积与粒数（粳稻为0.85，籼稻为0.82）和粒质量（粳稻为0.80，籼稻为0.76）高度相关。该方法为水稻穗高通量田间表型分析提供了一种低成本解决方案，加快了水稻育种效率。
