Montesinos-López Osval A, Chavira-Flores Moises, Crespo-Herrera Leo, Saint Piere Carolina, Li HuiHui, Fritsche-Neto Roberto, Al-Nowibet Khalid, Montesinos-López Abelardo, Crossa José
Facultad de Telemática, Universidad de Colima, Colima, Colima 28040, México.
Instituto de Investigaciones en Matemáticas Aplicadas y Sistemas (IIMAS), Universidad Nacional Autónoma de México (UNAM), Ciudad de México 04510, México.
Genetics. 2024 Nov 5;228(4). doi: 10.1093/genetics/iyae161.
Deep learning methods have been applied when working to enhance the prediction accuracy of traditional statistical methods in the field of plant breeding. Although deep learning seems to be a promising approach for genomic prediction, it has proven to have some limitations, since its conventional methods fail to leverage all available information. Multimodal deep learning methods aim to improve the predictive power of their unimodal counterparts by introducing several modalities (sources) of input information. In this review, we introduce some theoretical basic concepts of multimodal deep learning and provide a list of the most widely used neural network architectures in deep learning, as well as the available strategies to fuse data from different modalities. We mention some of the available computational resources for the practical implementation of multimodal deep learning problems. We finally performed a review of applications of multimodal deep learning to genomic selection in plant breeding and other related fields. We present a meta-picture of the practical performance of multimodal deep learning methods to highlight how these tools can help address complex problems in the field of plant breeding. We discussed some relevant considerations that researchers should keep in mind when applying multimodal deep learning methods. Multimodal deep learning holds significant potential for various fields, including genomic selection. While multimodal deep learning displays enhanced prediction capabilities over unimodal deep learning and other machine learning methods, it demands more computational resources. Multimodal deep learning effectively captures intermodal interactions, especially when integrating data from different sources. To apply multimodal deep learning in genomic selection, suitable architectures and fusion strategies must be chosen. It is relevant to keep in mind that multimodal deep learning, like unimodal deep learning, is a powerful tool but should be carefully applied. Given its predictive edge over traditional methods, multimodal deep learning is valuable in addressing challenges in plant breeding and food security amid a growing global population.
在努力提高植物育种领域传统统计方法的预测准确性时，深度学习方法已得到应用。尽管深度学习似乎是基因组预测的一种很有前景的方法，但事实证明它存在一些局限性，因为其传统方法未能充分利用所有可用信息。多模态深度学习方法旨在通过引入多种输入信息模态（来源）来提高其单模态对应方法的预测能力。在本综述中，我们介绍了多模态深度学习的一些理论基本概念，列出了深度学习中最广泛使用的神经网络架构，以及融合来自不同模态数据的可用策略。我们提到了一些用于实际解决多模态深度学习问题的可用计算资源。我们最后对多模态深度学习在植物育种和其他相关领域的基因组选择中的应用进行了综述。我们展示了多模态深度学习方法实际性能的全景，以突出这些工具如何有助于解决植物育种领域的复杂问题。我们讨论了研究人员在应用多模态深度学习方法时应牢记的一些相关注意事项。多模态深度学习在包括基因组选择在内的各个领域都具有巨大潜力。虽然多模态深度学习比单模态深度学习和其他机器学习方法具有更强的预测能力，但它需要更多的计算资源。多模态深度学习能有效捕捉模态间的相互作用，尤其是在整合来自不同来源的数据时。要在基因组选择中应用多模态深度学习，必须选择合适的架构和融合策略。需要记住的是，与单模态深度学习一样，多模态深度学习是一个强大的工具，但应用时应谨慎。鉴于其相对于传统方法的预测优势，多模态深度学习在应对全球人口增长背景下植物育种和粮食安全方面的挑战时具有重要价值。
