IEEE Trans Neural Syst Rehabil Eng. 2022;30:1103-1112. doi: 10.1109/TNSRE.2022.3168796. Epub 2022 May 3.
Predicting the user's intended locomotion mode is critical for wearable robot control to assist the user's seamless transitions when walking on changing terrains. Although machine vision has recently proven to be a promising tool in identifying upcoming terrains in the travel path, existing approaches are limited to environment perception rather than human intent recognition that is essential for coordinated wearable robot operation. Hence, in this study, we aim to develop a novel system that fuses the human gaze (representing user intent) and machine vision (capturing environmental information) for accurate prediction of the user's locomotion mode. The system possesses multimodal visual information and recognizes user's locomotion intent in a complex scene, where multiple terrains are present. Additionally, based on the dynamic time warping algorithm, a fusion strategy was developed to align temporal predictions from individual modalities while producing flexible decisions on the timing of locomotion mode transition for wearable robot control. System performance was validated using experimental data collected from five participants, showing high accuracy (over 96% in average) of intent recognition and reliable decision-making on locomotion transition with adjustable lead time. The promising results demonstrate the potential of fusing human gaze and machine vision for locomotion intent recognition of lower limb wearable robots.
预测用户的预期运动模式对于可穿戴机器人控制至关重要，可帮助用户在变化的地形上实现无缝过渡。尽管机器视觉最近已被证明是识别行进路径中即将出现的地形的一种有前途的工具，但现有的方法仅限于环境感知，而无法识别对协调可穿戴机器人操作至关重要的人类意图。因此，在本研究中，我们旨在开发一种新系统，该系统融合了人类注视（代表用户意图）和机器视觉（捕捉环境信息），以准确预测用户的运动模式。该系统具有多模态视觉信息，可以识别复杂场景中（存在多种地形）用户的运动意图。此外，基于动态时间规整算法，开发了一种融合策略，以对齐来自各个模态的时间预测，同时为可穿戴机器人控制的运动模式转换时间做出灵活决策。使用从五名参与者收集的实验数据验证了系统性能，结果表明意图识别的准确率很高（平均超过 96%），并且在可穿戴机器人控制的运动模式转换方面能够可靠地做出决策，并可调整前置时间。有前景的结果表明，融合人类注视和机器视觉对于下肢可穿戴机器人的运动意图识别具有潜力。
