Department of Electrical and Computer Engineering, Ajou University, Suwon-si 16499, Republic of Korea.
Graduate School of Convergence Science and Technology, RICS, Seoul National University, Seoul 08826, Republic of Korea.
Sensors (Basel). 2023 Jun 2;23(11):5304. doi: 10.3390/s23115304.
With the advancement of computer hardware and communication technologies, deep learning technology has made significant progress, enabling the development of systems that can accurately estimate human emotions. Factors such as facial expressions, gender, age, and the environment influence human emotions, making it crucial to understand and capture these intricate factors. Our system aims to recommend personalized images by accurately estimating human emotions, age, and gender in real time. The primary objective of our system is to enhance user experiences by recommending images that align with their current emotional state and characteristics. To achieve this, our system collects environmental information, including weather conditions and user-specific environment data through APIs and smartphone sensors. Additionally, we employ deep learning algorithms for real-time classification of eight types of facial expressions, age, and gender. By combining this facial information with the environmental data, we categorize the user's current situation into positive, neutral, and negative stages. Based on this categorization, our system recommends natural landscape images that are colorized using Generative Adversarial Networks (GANs). These recommendations are personalized to match the user's current emotional state and preferences, providing a more engaging and tailored experience. Through rigorous testing and user evaluations, we assessed the effectiveness and user-friendliness of our system. Users expressed satisfaction with the system's ability to generate appropriate images based on the surrounding environment, emotional state, and demographic factors such as age and gender. The visual output of our system significantly impacted users' emotional responses, resulting in a positive mood change for most users. Moreover, the system's scalability was positively received, with users acknowledging its potential benefits when installed outdoors and expressing a willingness to continue using it. Compared to other recommender systems, our integration of age, gender, and weather information provides personalized recommendations, contextual relevance, increased engagement, and a deeper understanding of user preferences, thereby enhancing the overall user experience. The system's ability to comprehend and capture intricate factors that influence human emotions holds promise in various domains, including human-computer interaction, psychology, and social sciences.
随着计算机硬件和通信技术的进步，深度学习技术取得了重大进展，使得能够开发出能够准确估计人类情绪的系统。面部表情、性别、年龄和环境等因素影响着人类的情绪，因此理解和捕捉这些复杂的因素至关重要。我们的系统旨在通过实时准确地估计人类情绪、年龄和性别来推荐个性化的图像。我们系统的主要目标是通过推荐与当前情绪状态和特征相匹配的图像来增强用户体验。为了实现这一目标，我们的系统通过 API 和智能手机传感器收集环境信息，包括天气条件和用户特定的环境数据。此外，我们还采用深度学习算法实时对八种类型的面部表情、年龄和性别进行分类。通过将这些面部信息与环境数据相结合，我们将用户的当前情况分为积极、中性和消极三个阶段。基于这种分类，我们的系统推荐使用生成对抗网络 (GAN) 进行色彩化的自然景观图像。这些推荐是个性化的，以匹配用户当前的情绪状态和偏好，提供更具吸引力和个性化的体验。通过严格的测试和用户评估，我们评估了我们系统的有效性和用户友好性。用户对系统根据周围环境、情绪状态以及年龄和性别等人口统计因素生成适当图像的能力表示满意。我们系统的视觉输出对用户的情绪反应产生了显著影响，大多数用户的情绪都发生了积极的变化。此外，系统的可扩展性也受到了积极的评价，用户承认其在户外安装时的潜在好处，并表示愿意继续使用它。与其他推荐系统相比，我们整合年龄、性别和天气信息提供个性化推荐、上下文相关性、增加参与度以及更深入地了解用户偏好，从而提升整体用户体验。该系统理解和捕捉影响人类情绪的复杂因素的能力在人机交互、心理学和社会科学等各个领域具有广阔的应用前景。
