Faculty of Engineering, Universidad de Antioquia, Medellín, Antioquia, Colombia.
School of Engineering, Universidad Autónoma de Madrid, Madrid, Madrid, España.
PLoS One. 2023 Feb 2;18(2):e0281248. doi: 10.1371/journal.pone.0281248. eCollection 2023.
Patients suffering from Parkinson's disease (PD) present a reduction in facial movements called hypomimia. In this work, we propose to use machine learning facial expression analysis from face images based on action unit domains to improve PD detection. We propose different domain adaptation techniques to exploit the latest advances in automatic face analysis and face action unit detection.
Three different approaches are explored to model facial expressions of PD patients: (i) face analysis using single frame images and also using sequences of images, (ii) transfer learning from face analysis to action units recognition, and (iii) triplet-loss functions to improve the automatic classification between patients and healthy subjects.
Real face images from PD patients show that it is possible to properly model elicited facial expressions using image sequences (neutral, onset-transition, apex, offset-transition, and neutral) with accuracy improvements of up to 5.5% (from 72.9% to 78.4%) with respect to single-image PD detection. We also show that our proposed action unit domain adaptation provides improvements of up to 8.9% (from 78.4% to 87.3%) with respect to face analysis. Finally, we also show that triplet-loss functions provide improvements of up to 3.6% (from 78.8% to 82.4%) with respect to action unit domain adaptation applied upon models created from scratch. The code of the experiments is available at https://github.com/luisf-gomez/Explorer-FE-AU-in-PD.
Domain adaptation via transfer learning methods seem to be a promising strategy to model hypomimia in PD patients. Considering the good results and also the fact that only up to five images per participant are considered in each sequence, we believe that this work is a step forward in the development of inexpensive computational systems suitable to model and quantify problems of PD patients in their facial expressions.
帕金森病（PD）患者会出现面部运动减少，即运动不能。在这项工作中，我们建议使用基于动作单元域的机器学习面部表情分析来改善 PD 的检测。我们提出了不同的领域自适应技术，以利用自动面部分析和面部动作单元检测的最新进展。
我们探索了三种不同的方法来对 PD 患者的面部表情进行建模：（i）使用单帧图像和图像序列进行面部分析，（ii）从面部分析到动作单元识别的迁移学习，以及（iii）使用三元组损失函数来改善患者和健康受试者之间的自动分类。
PD 患者的真实面部图像表明，使用图像序列（中性、起始-转换、顶点、结束-转换和中性）可以正确地对诱发的面部表情进行建模，与单一图像 PD 检测相比，准确率提高了 5.5%（从 72.9%提高到 78.4%）。我们还表明，我们提出的动作单元域自适应方法提供了高达 8.9%的改进（从 78.4%提高到 87.3%），与从头开始创建的模型上应用的面部分析相比。实验的代码可在 https://github.com/luisf-gomez/Explorer-FE-AU-in-PD 上获得。
通过迁移学习方法的领域自适应似乎是一种很有前途的策略，可以对 PD 患者的运动不能进行建模。考虑到良好的结果，以及每个序列中每个参与者仅考虑五张图像的事实，我们相信这项工作是朝着开发适合于对 PD 患者面部表情进行建模和量化的廉价计算系统迈出的一步。
