Yiu Chunki, Liu Yiming, Park Wooyoung, Li Jian, Huang Xingcan, Yao Kuanming, Gao Yuyu, Zhao Guangyao, Chu Hongwei, Zhou Jingkun, Li Dengfeng, Li Hu, Zhang Binbin, Chow Lung, Huang Ya, Xu Qingsong, Yu Xinge
Department of Biomedical Engineering, City University of Hong Kong, Hong Kong, China.
Hong Kong Center for Cerebra-Cardiovascular Health Engineering, Hong Kong Science Park, New Territories, Hong Kong, China.
Sci Adv. 2025 Mar 28;11(13):eadt6041. doi: 10.1126/sciadv.adt6041. Epub 2025 Mar 26.
Unmanned aerial vehicles have undergone substantial development and market growth recently. With research focusing on improving control strategies for better user experience, feedback systems, which are vital for operator awareness of surroundings and flight status, remain underdeveloped. Current bulky manipulators also hinder accuracy and usability. Here, we present an enhanced human-machine interface based on skin-integrated multimodal sensing and feedback devices for closed-loop drone control. This system captures hand gestures for intuitive, rapid, and precise control. An integrated tactile actuator array translates the drone's posture into two-dimensional tactile information, enhancing the operator's perception of the flight situation. Integrated obstacle detection and neuromuscular electrical stimulation-based force feedback system enable collision avoidance and flight path correction. This closed-loop system combines intuitive controls and multimodal feedback to reduce training time and cognitive load while improving flight stability, environmental awareness, and the drone's posture. The use of stretchable electronics also addresses wearability and bulkiness issues in traditional systems, advancing human-machine interface design.
无人机近年来取得了长足的发展，市场也不断增长。随着研究重点转向改进控制策略以提升用户体验，对于操作人员了解周围环境和飞行状态至关重要的反馈系统仍未得到充分发展。当前笨重的操纵器也妨碍了准确性和可用性。在此，我们展示了一种基于皮肤集成多模态传感和反馈设备的增强型人机界面，用于无人机的闭环控制。该系统捕捉手势以实现直观、快速且精确的控制。集成的触觉致动器阵列将无人机的姿态转化为二维触觉信息，增强操作人员对飞行状况的感知。集成的障碍物检测和基于神经肌肉电刺激的力反馈系统能够实现避障和飞行路径校正。这种闭环系统将直观控制与多模态反馈相结合，减少训练时间和认知负荷，同时提高飞行稳定性、环境感知能力以及无人机的姿态。可拉伸电子器件的使用还解决了传统系统中的可穿戴性和笨重问题，推动了人机界面设计的发展。
