Univ. Bourgogne Franche-Comté, ImViA EA7535, Dijon, France.
Biomed Eng Online. 2022 Sep 19;21(1):69. doi: 10.1186/s12938-022-01037-z.
Remote photoplethysmography (rPPG) is a technique developed to estimate heart rate using standard video cameras and ambient light. Due to the multiple sources of noise that deteriorate the quality of the signal, conventional filters such as the bandpass and wavelet-based filters are commonly used. However, after using conventional filters, some alterations remain, but interestingly an experienced eye can easily identify them.
We studied a long short-term memory (LSTM) network in the rPPG filtering task to identify these alterations using many-to-one and many-to-many approaches. We used three public databases in intra-dataset and cross-dataset scenarios, along with different protocols to analyze the performance of the method. We demonstrate how the network can be easily trained with a set of 90 signals totaling around 45 min. On the other hand, we show the stability of the LSTM performance with six state-of-the-art rPPG methods.
This study demonstrates the superiority of the LSTM-based filter experimentally compared with conventional filters in an intra-dataset scenario. For example, we obtain on the VIPL database an MAE of 3.9 bpm, whereas conventional filtering improves performance on the same dataset from 10.3 bpm to 7.7 bpm. The cross-dataset approach presents a dependence in the network related to the average signal-to-noise ratio on the rPPG signals, where the closest signal-to-noise ratio values in the training and testing set the better. Moreover, it was demonstrated that a relatively small amount of data are sufficient to successfully train the network and outperform the results obtained by classical filters. More precisely, we have shown that about 45 min of rPPG signal could be sufficient to train an effective LSTM deep-filter.
远程光体积描记术（rPPG）是一种使用标准摄像机和环境光估算心率的技术。由于有多种噪声源会降低信号质量，因此通常使用带通和基于小波的滤波器等常规滤波器。但是，使用常规滤波器后，仍会存在一些变化，但是有趣的是，有经验的人可以轻松识别它们。
我们在 rPPG 滤波任务中研究了长短期记忆（LSTM）网络，以使用多对一和多对多方法识别这些变化。我们在内部数据集和跨数据集场景中使用了三个公共数据库，以及不同的协议来分析该方法的性能。我们展示了如何使用总共约 45 分钟的 90 个信号集轻松训练网络。另一方面，我们展示了 LSTM 性能在六个最先进的 rPPG 方法中的稳定性。
与内部数据集场景中的传统滤波器相比，本研究通过实验证明了基于 LSTM 的滤波器的优越性。例如，我们在 VIPL 数据库中获得了 3.9 bpm 的 MAE，而常规滤波可将同一数据集的性能从 10.3 bpm 提高到 7.7 bpm。跨数据集方法表现出与 rPPG 信号的平均信噪比有关的网络依赖性，其中训练集和测试集的信号与平均信噪比值越接近，性能越好。此外，证明了少量数据就足以成功训练网络，并超过经典滤波器获得的结果。更准确地说，我们已经表明，大约 45 分钟的 rPPG 信号就足以训练有效的 LSTM 深度滤波器。
