Corcoran Evangeline, Hosseini Kasra, Siles Laura, Kurup Smita, Ahnert Sebastian
The Alan Turing Institute, London, United Kingdom.
Zalando Societas Europaea (SE), Berlin, Germany.
Front Plant Sci. 2025 May 22;16:1443882. doi: 10.3389/fpls.2025.1443882. eCollection 2025.
Recent advancements in sensor technologies have enabled collection of many large, high-resolution plant images datasets that could be used to non-destructively explore the relationships between genetics, environment and management factors on phenotype or the physical traits exhibited by plants. The phenotype data captured in these datasets could then be integrated into models of plant development and crop yield to more accurately predict how plants may grow as a result of changing management practices and climate conditions, better ensuring future food security. However, automated methods capable of reliably and efficiently extracting meaningful measurements of individual plant components (e.g. leaves, flowers, pods) from imagery of whole plants are currently lacking. In this study, we explore interdisciplinary application of MapReader, a computer vision pipeline for annotating and classifying patches of larger images that was originally developed for semantic exploration of historical maps, to time-series images of whole oilseed rape () plants.
Models were trained to classify five plant structures in patches derived from whole plant images (branches, leaves, pods, flower buds and flowers), as well as background patches. Three modelling methods are compared: (i) 6-label multi-class classification, (ii) a chain of binary classifiers approach, and (iii) an approach combining binary classification of plant and background patches, followed by 5-label multi-class classification of plant structures.
A combined plant/background binarization and 5-label multi-class modelling approach using a 'resnext50d_4s2x40d' model architecture for both the binary classification and multi-class classification components was found to produce the most accurate patch classification for whole plant images (macro-averaged F1-score = 88.50, weighted average F1-score = 97.71). This combined binary and 5-label multi-class classification approach demonstrate similar performance to the top-performing MapReader 'railspace' classification model.
This highlights the potential applicability of the MapReader model framework to images data from across scientific and humanities domains, and the flexibility it provides in creating pipelines with different modelling approaches. The pipeline for dynamic plant phenotyping from whole plant images developed in this study could potentially be applied to imagery from varied laboratory conditions, and to images datasets of other plants of both agricultural and conservation concern.
传感器技术的最新进展使得能够收集许多大型、高分辨率的植物图像数据集，这些数据集可用于无损探索遗传学、环境和管理因素与植物表型或植物所表现出的物理特征之间的关系。然后，这些数据集中捕获的表型数据可以整合到植物发育和作物产量模型中，以更准确地预测由于管理实践和气候条件变化，植物可能如何生长，从而更好地确保未来的粮食安全。然而，目前缺乏能够从整株植物图像中可靠且高效地提取单个植物组成部分（如叶子、花朵、豆荚）有意义测量值的自动化方法。在本研究中，我们探索了MapReader的跨学科应用，MapReader是一种计算机视觉管道，用于注释和分类较大图像的补丁，最初是为历史地图的语义探索而开发的，现在应用于全株油菜的时间序列图像。
训练模型对从整株植物图像中提取的补丁中的五种植物结构（树枝、叶子、豆荚、花芽和花朵）以及背景补丁进行分类。比较了三种建模方法：（i）6标签多类分类，（ii）二元分类器链方法，以及（iii）一种方法，先对植物和背景补丁进行二元分类，然后对植物结构进行5标签多类分类。
发现一种结合植物/背景二值化和5标签多类建模方法，在二元分类和多类分类组件中都使用“resnext50d_4s2x40d”模型架构，对全株植物图像产生最准确的补丁分类（宏平均F1分数 = 88.50，加权平均F1分数 = 97.71）。这种结合二元和5标签多类分类方法的性能与表现最佳的MapReader“railspace”分类模型相似。
这突出了MapReader模型框架对来自科学和人文领域的图像数据的潜在适用性，以及它在创建具有不同建模方法的管道时提供的灵活性。本研究中开发的用于从全株植物图像进行动态植物表型分析的管道可能潜在地应用于各种实验室条件下的图像，以及其他具有农业和保护意义的植物的图像数据集。
