Amangeldy Nurzada, Gazizova Nazerke, Milosz Marek, Kurmetbek Bekbolat, Nazyrova Aizhan, Kassymova Akmaral
Faculty of Information Technologies, L.N. Gumilyov Eurasian National University, Astana 010008, Kazakhstan.
Department of Computer Science, Lublin University of Technology, 36B Nadbystrzycka Str., 20-618 Lublin, Poland.
Sensors (Basel). 2025 Jun 13;25(12):3703. doi: 10.3390/s25123703.
This paper presents a new methodology for analyzing lip articulation during fingerspelling aimed at extracting robust visual patterns that can overcome the inherent ambiguity and variability of lip shape. The proposed approach is based on unsupervised clustering of lip movement trajectories to identify consistent articulatory patterns across different time profiles. The methodology is not limited to using a single model. Still, it includes the exploration of varying cluster configurations and an assessment of their robustness, as well as a detailed analysis of the correspondence between individual alphabet letters and specific clusters. In contrast to direct classification based on raw visual features, this approach pre-tests clustered representations using a model-based assessment of their discriminative potential. This structured approach enhances the interpretability and robustness of the extracted features, highlighting the importance of lip dynamics as an auxiliary modality in multimodal sign language recognition. The obtained results demonstrate that trajectory clustering can serve as a practical method for generating features, providing more accurate and context-sensitive gesture interpretation.
本文提出了一种新的方法，用于分析手指拼写过程中的唇部发音，旨在提取能够克服唇部形状固有模糊性和变异性的稳健视觉模式。所提出的方法基于唇部运动轨迹的无监督聚类，以识别不同时间轮廓上一致的发音模式。该方法不限于使用单一模型，还包括探索不同的聚类配置及其稳健性评估，以及对单个字母与特定聚类之间对应关系的详细分析。与基于原始视觉特征的直接分类不同，该方法使用基于模型的判别潜力评估对聚类表示进行预测试。这种结构化方法增强了所提取特征的可解释性和稳健性，突出了唇部动态作为多模态手语识别辅助模态的重要性。所得结果表明，轨迹聚类可作为一种生成特征的实用方法，提供更准确且上下文敏感的手势解释。
