University of Zagreb, Faculty of Electrical Engineering and Computing, Unska 3, 10000 Zagreb, Croatia.
Psychiatr Danub. 2013 Sep;25(3):340-6.
Contemporary psychiatry is looking at affective sciences to understand human behavior, cognition and the mind in health and disease. Since it has been recognized that emotions have a pivotal role for the human mind, an ever increasing number of laboratories and research centers are interested in affective sciences, affective neuroscience, affective psychology and affective psychopathology. Therefore, this paper presents multidisciplinary research results of Laboratory for Interactive Simulation System at Faculty of Electrical Engineering and Computing, University of Zagreb in the stress resilience. Patient's distortion in emotional processing of multimodal input stimuli is predominantly consequence of his/her cognitive deficit which is result of their individual mental health disorders. These emotional distortions in patient's multimodal physiological, facial, acoustic, and linguistic features related to presented stimulation can be used as indicator of patient's mental illness. Real-time processing and analysis of patient's multimodal response related to annotated input stimuli is based on appropriate machine learning methods from computer science. Comprehensive longitudinal multimodal analysis of patient's emotion, mood, feelings, attention, motivation, decision-making, and working memory in synchronization with multimodal stimuli provides extremely valuable big database for data mining, machine learning and machine reasoning. Presented multimedia stimuli sequence includes personalized images, movies and sounds, as well as semantically congruent narratives. Simultaneously, with stimuli presentation patient provides subjective emotional ratings of presented stimuli in terms of subjective units of discomfort/distress, discrete emotions, or valence and arousal. These subjective emotional ratings of input stimuli and corresponding physiological, speech, and facial output features provides enough information for evaluation of patient's cognitive appraisal deficit. Aggregated real-time visualization of this information provides valuable assistance in patient mental state diagnostics enabling therapist deeper and broader insights into dynamics and progress of the psychotherapy.
当代精神病学正在关注情感科学，以了解健康和疾病中的人类行为、认知和思维。由于人们已经认识到情绪对人类思维起着关键作用，越来越多的实验室和研究中心对情感科学、情感神经科学、情感心理学和情感精神病理学感兴趣。因此，本文介绍了萨格勒布大学电气与计算机工程学院交互模拟系统实验室在应激弹性方面的多学科研究成果。患者对多模态输入刺激的情绪处理的扭曲主要是由于其认知缺陷，而认知缺陷是其个体心理健康障碍的结果。患者在与呈现刺激相关的多模态生理、面部、声学和语言特征中的这些情绪扭曲可以用作患者精神疾病的指标。基于计算机科学的适当机器学习方法，实时处理和分析患者与注释输入刺激相关的多模态反应。与多模态刺激同步对患者的情绪、情绪、感觉、注意力、动机、决策和工作记忆进行综合纵向多模态分析，为数据挖掘、机器学习和机器推理提供了极其有价值的大数据集。呈现的多媒体刺激序列包括个性化的图像、电影和声音，以及语义一致的叙述。同时，在呈现刺激的同时，患者会根据主观不适/痛苦、离散情绪或效价和唤醒的主观单位，对呈现的刺激进行主观情绪评分。这些输入刺激的主观情绪评分以及相应的生理、语音和面部输出特征提供了足够的信息，可用于评估患者的认知评估缺陷。实时聚合可视化此信息提供了有价值的帮助，可用于患者的精神状态诊断，使治疗师能够更深入、更广泛地了解心理治疗的动态和进展。
