School of Computation, Information and Technology, Technische Universität München (TUM), Arcisstr. 21, 80333 München, Germany.
MIT Center for Collective Intelligence, 245 First St., E94-1509, Cambridge, MA 02142, USA.
Sensors (Basel). 2024 Mar 16;24(6):1917. doi: 10.3390/s24061917.
Recent advances in artificial intelligence combined with behavioral sciences have led to the development of cutting-edge tools for recognizing human emotions based on text, video, audio, and physiological data. However, these data sources are expensive, intrusive, and regulated, unlike plants, which have been shown to be sensitive to human steps and sounds. A methodology to use plants as human emotion detectors is proposed. Electrical signals from plants were tracked and labeled based on video data. The labeled data were then used for classification., and the MLP, biLSTM, MFCC-CNN, MFCC-ResNet, Random Forest, 1-Dimensional CNN, and biLSTM (without windowing) models were set using a grid search algorithm with cross-validation. Finally, the best-parameterized models were trained and used on the test set for classification. The performance of this methodology was measured via a case study with 54 participants who were watching an emotionally charged video; as ground truth, their facial emotions were simultaneously measured using facial emotion analysis. The Random Forest model shows the best performance, particularly in recognizing high-arousal emotions, achieving an overall weighted accuracy of 55.2% and demonstrating high weighted recall in emotions such as fear (61.0%) and happiness (60.4%). The MFCC-ResNet model offers decently balanced results, with AccuracyMFCC-ResNet=0.318 and RecallMFCC-ResNet=0.324. Regarding the MFCC-ResNet model, fear and anger were recognized with 75% and 50% recall, respectively. Thus, using plants as an emotion recognition tool seems worth investigating, addressing both cost and privacy concerns.
人工智能与行为科学的最新进展使得能够基于文本、视频、音频和生理数据来开发用于识别人类情感的尖端工具。然而，这些数据源与植物不同，它们昂贵、具有侵入性且受到监管，而植物已被证明对人类的脚步声和声音敏感。提出了一种使用植物作为人类情感探测器的方法。根据视频数据跟踪和标记植物的电信号。然后使用标记的数据进行分类，并使用网格搜索算法和交叉验证设置 MLP、biLSTM、MFCC-CNN、MFCC-ResNet、随机森林、1 维 CNN 和 biLSTM（无窗口）模型。最后，对最佳参数化模型进行训练并用于分类测试集。通过对 54 名观看情感视频的参与者进行案例研究来衡量该方法的性能；作为地面实况，同时使用面部情绪分析来测量他们的面部情绪。随机森林模型表现出最佳性能，特别是在识别高唤醒情绪方面，整体加权准确率达到 55.2%，在恐惧（61.0%）和幸福（60.4%）等情绪方面具有较高的加权召回率。MFCC-ResNet 模型提供了相当平衡的结果，AccuracyMFCC-ResNet=0.318 和 RecallMFCC-ResNet=0.324。关于 MFCC-ResNet 模型，恐惧和愤怒的召回率分别为 75%和 50%。因此，使用植物作为情感识别工具似乎值得研究，既解决了成本问题，也解决了隐私问题。
