A client-server based recognition system: Non-contact single/multiple emotional and behavioral state assessment methods.

BACKGROUND AND OBJECTIVES

In the current global health landscape, there is an increasing demand for rapid and accurate assessment of mental states. Traditional assessment methods typically rely on face-to-face interactions, which are not only time-consuming but also highly subjective. Addressing this issue, this study aims to develop a client-server-based, non-contact multimodal emotion and behavior recognition system to enhance the efficiency and accuracy of mental state assessments.

METHODS

This study designed and implemented a multimodal assessment system integrating voice, text, facial expressions, and body movements. Utilizing a client-server architecture, the system optimizes diagnostic efficiency and decision-making accuracy through an intuitive visual interface. The system's effectiveness was validated and tested in actual hospital settings.

RESULTS

The system demonstrated exceptional performance in multimodal emotion and behavior recognition, achieving a voice recognition accuracy of 92.01%, facial expression recognition accuracy of 91.3%, and an overall multimodal assessment accuracy of 77.9%. Moreover, it reached a behavior analysis accuracy of 94.5%.

CONCLUSIONS

The multimodal assessment system developed in this study significantly enhances the accuracy and efficiency of mental state assessments, meeting the needs of clinicians for precise and rapid diagnostics in real-world settings.