Xu Siyuan, Zhang Sunjie
School of Optical-Electrical and Computer Engineering, University of Shanghai for Science and Technology, Shanghai 200093, P. R. China.
Sheng Wu Yi Xue Gong Cheng Xue Za Zhi. 2025 Jun 25;42(3):552-559. doi: 10.7507/1001-5515.202408026.
Multi-task learning (MTL) has demonstrated significant advantages in the field of physiological signal measurement. This approach enhances the model's generalization ability by sharing parameters and features between similar tasks, even in data-scarce environments. However, traditional multi-task physiological signal measurement methods face challenges such as feature conflicts between tasks, task imbalance, and excessive model complexity, which limit their application in complex environments. To address these issues, this paper proposes an enhanced multi-scale spatiotemporal network (EMSTN) based on Eulerian video magnification (EVM), super-resolution reconstruction and convolutional multilayer perceptron. First, EVM is introduced in the input stage of the network to amplify subtle color and motion changes in the video, significantly improving the model's ability to capture pulse and respiratory signals. Additionally, a super-resolution reconstruction module is integrated into the network to enhance the image resolution, thereby improving detail capture and increasing the accuracy of facial action unit (AU) tasks. Then, convolutional multilayer perceptron is employed to replace traditional 2D convolutions, improving feature extraction efficiency and flexibility, which significantly boosts the performance of heart rate and respiratory rate measurements. Finally, comprehensive experiments on the Binghamton-Pittsburgh 4D Spontaneous Facial Expression Database (BP4D+) fully validate the effectiveness and superiority of the proposed method in multi-task physiological signal measurement.
多任务学习（MTL）在生理信号测量领域已展现出显著优势。这种方法通过在相似任务间共享参数和特征来增强模型的泛化能力，即便在数据稀缺的环境中亦是如此。然而，传统的多任务生理信号测量方法面临诸如任务间特征冲突、任务不平衡以及模型复杂度过高之类的挑战，这些限制了它们在复杂环境中的应用。为解决这些问题，本文提出一种基于欧拉视频放大（EVM）、超分辨率重建和卷积多层感知器的增强型多尺度时空网络（EMSTN）。首先，在网络的输入阶段引入EVM以放大视频中细微的颜色和运动变化，显著提升模型捕捉脉搏和呼吸信号的能力。此外，将一个超分辨率重建模块集成到网络中以提高图像分辨率，从而改善细节捕捉并提高面部动作单元（AU）任务的准确性。然后，采用卷积多层感知器来取代传统的二维卷积，提高特征提取效率和灵活性，这显著提升了心率和呼吸率测量的性能。最后，在宾厄姆顿 - 匹兹堡4D自发面部表情数据库（BP4D +）上进行的综合实验充分验证了所提方法在多任务生理信号测量中的有效性和优越性。
