Pinto José, Ramos João R C, Costa Rafael S, Rossell Sergio, Dumas Patrick, Oliveira Rui
LAQV-REQUIMTE, Department of Chemistry, NOVA School of Science and Technology, NOVA University Lisbon, Caparica, Portugal.
GlaxoSmithKline, Rixensart, Belgium.
Front Bioeng Biotechnol. 2023 Sep 8;11:1237963. doi: 10.3389/fbioe.2023.1237963. eCollection 2023.
Hybrid modeling combining First-Principles with machine learning is becoming a pivotal methodology for Biopharma 4.0 enactment. Chinese Hamster Ovary (CHO) cells, being the workhorse for industrial glycoproteins production, have been the object of several hybrid modeling studies. Most previous studies pursued a shallow hybrid modeling approach based on three-layered Feedforward Neural Networks (FFNNs) combined with macroscopic material balance equations. Only recently, the hybrid modeling field is incorporating deep learning into its framework with significant gains in descriptive and predictive power. This study compares, for the first time, deep and shallow hybrid modeling in a CHO process development context. Data of 24 fed-batch cultivations of a CHO-K1 cell line expressing a target glycoprotein, comprising 30 measured state variables over time, were used to compare both methodologies. Hybrid models with varying FFNN depths (3-5 layers) were systematically compared using two training methodologies. The classical training is based on the Levenberg-Marquardt algorithm, indirect sensitivity equations and cross-validation. The deep learning is based on the Adaptive Moment Estimation Method (ADAM), stochastic regularization and semidirect sensitivity equations. The results point to a systematic generalization improvement of deep hybrid models over shallow hybrid models. Overall, the training and testing errors decreased by 14.0% and 23.6% respectively when applying the deep methodology. The Central Processing Unit (CPU) time for training the deep hybrid model increased by 31.6% mainly due to the higher FFNN complexity. The final deep hybrid model is shown to predict the dynamics of the 30 state variables within the error bounds in every test experiment. Notably, the deep hybrid model could predict the metabolic shifts in key metabolites (e.g., lactate, ammonium, glutamine and glutamate) in the test experiments. We expect deep hybrid modeling to accelerate the deployment of high-fidelity digital twins in the biopharma sector in the near future.
将第一性原理与机器学习相结合的混合建模正成为生物制药4.0实施的关键方法。中国仓鼠卵巢（CHO）细胞作为工业糖蛋白生产的主力军，已成为多项混合建模研究的对象。以前的大多数研究采用基于三层前馈神经网络（FFNN）与宏观物料平衡方程相结合的浅层混合建模方法。直到最近，混合建模领域才将深度学习纳入其框架，在描述和预测能力方面有了显著提升。本研究首次在CHO工艺开发背景下比较了深度和浅层混合建模。使用表达目标糖蛋白的CHO-K1细胞系的24次补料分批培养数据，包括随时间测量的30个状态变量，来比较这两种方法。使用两种训练方法系统地比较了具有不同FFNN深度（3 - 5层）的混合模型。经典训练基于Levenberg-Marquardt算法、间接灵敏度方程和交叉验证。深度学习基于自适应矩估计方法（ADAM）、随机正则化和半直接灵敏度方程。结果表明，深度混合模型比浅层混合模型在系统泛化方面有改进。总体而言，应用深度方法时，训练和测试误差分别降低了14.0%和23.6%。训练深度混合模型的中央处理器（CPU）时间增加了31.6%，主要是由于FFNN复杂度更高。最终的深度混合模型在每个测试实验中都能在误差范围内预测30个状态变量的动态。值得注意的是，深度混合模型能够在测试实验中预测关键代谢物（如乳酸、铵、谷氨酰胺和谷氨酸）的代谢变化。我们预计深度混合建模将在不久的将来加速高保真数字孪生在生物制药领域的应用。
