Shanghai University, School of Computer Engineering and Science, Shanghai, 200444, China.
Loughborough University, Wolfson School of Mechanical, Electrical and Manufacturing Engineering, Loughborough, Leicestershire, LE11 3TU, UK.
Sci Rep. 2017 Aug 1;7(1):7081. doi: 10.1038/s41598-017-07122-x.
Recent progress in Affective Computing (AC) has enabled integration of physiological cues and spontaneous expressions to reveal a subject's emotional state. Due to the lack of an effective technique for evaluating multimodal correlations, experience and intuition play a main role in present AC studies when fusing affective cues or modalities, resulting in unexpected outcomes. This study seeks to demonstrate a dynamic correlation between two such affective cues, physiological changes and spontaneous expressions, which were obtained by a combination of stereo vision based tracking and imaging photoplethysmography (iPPG), with a designed protocol involving 20 healthy subjects. The two cues obtained were sampled into a Statistical Association Space (SAS) to evaluate their dynamic correlation. It is found that the probability densities in the SAS increase as the peaks in two cues are approached. Also the complex form of the high probability density region in the SAS suggests a nonlinear correlation between two cues. Finally the cumulative distribution on the zero time-difference surface is found to be small (<0.047) demonstrating a lack of simultaneity. These results show that the two cues have a close interrelation, that is both asynchronous and nonlinear, in which a peak of one cue heralds a peak in the other.
情感计算 (AC) 的最新进展使得生理信号和自然表达能够被整合，以揭示主体的情绪状态。由于缺乏评估多模态相关性的有效技术，在当前的 AC 研究中，当融合情感线索或模态时，经验和直觉起着主要作用，导致了意想不到的结果。本研究旨在展示两种情感线索，即生理变化和自然表达之间的动态相关性，这些线索是通过基于立体视觉的跟踪和成像光体积描记术 (iPPG) 的组合获得的，采用了一项涉及 20 名健康受试者的设计方案。将获得的两个线索采样到统计关联空间 (SAS) 中，以评估它们的动态相关性。结果发现，随着两个线索的峰值接近，SAS 中的概率密度增加。此外，SAS 中高概率密度区域的复杂形式表明两个线索之间存在非线性相关性。最后，在零时间差表面上的累积分布被发现很小（<0.047），表明缺乏同时性。这些结果表明，这两个线索具有密切的相互关系，即两者都是异步和非线性的，一个线索的峰值预示着另一个线索的峰值。
