Xu Jun, Xu Shilong, Ma Mingyu, Ma Jing, Li Chuanlong
School of Automation, Harbin University of Science and Technology, Harbin 150080, China.
Artificial Intelligence Robot Joint Laboratory, Harbin University of Science and Technology, Harbin 150080, China.
Sensors (Basel). 2025 Jul 21;25(14):4518. doi: 10.3390/s25144518.
Blind and visually impaired (BVI) people face significant challenges in perception, navigation, and safety during travel. Existing infrastructure (e.g., blind lanes) and traditional aids (e.g., walking sticks, basic audio feedback) provide limited flexibility and interactivity for complex environments. To solve this problem, we propose a real-time travel assistance system based on deep learning. The hardware comprises an NVIDIA Jetson Nano controller, an Intel D435i depth camera for environmental sensing, and SG90 servo motors for feedback. To address embedded device computational constraints, we developed a lightweight object detection and segmentation algorithm. Key innovations include a multi-scale attention feature extraction backbone, a dual-stream fusion module incorporating the Mamba architecture, and adaptive context-aware detection/segmentation heads. This design ensures high computational efficiency and real-time performance. The system workflow is as follows: (1) the D435i captures real-time environmental data; (2) the processor analyzes this data, converting obstacle distances and path deviations into electrical signals; (3) servo motors deliver vibratory feedback for guidance and alerts. Preliminary tests confirm that the system can effectively detect obstacles and correct path deviations in real time, suggesting its potential to assist BVI users. However, as this is a work in progress, comprehensive field trials with BVI participants are required to fully validate its efficacy.
盲人及视障人士在出行过程中的感知、导航和安全方面面临重大挑战。现有的基础设施（如盲道）和传统辅助工具（如手杖、基本音频反馈）在复杂环境中提供的灵活性和交互性有限。为了解决这个问题，我们提出了一种基于深度学习的实时出行辅助系统。硬件包括一个英伟达Jetson Nano控制器、一个用于环境感知的英特尔D435i深度摄像头以及用于反馈的SG90伺服电机。为了应对嵌入式设备的计算限制，我们开发了一种轻量级目标检测与分割算法。关键创新包括一个多尺度注意力特征提取主干、一个结合曼巴架构的双流融合模块以及自适应上下文感知检测/分割头。这种设计确保了高计算效率和实时性能。系统工作流程如下：（1）D435i摄像头捕获实时环境数据；（2）处理器分析这些数据，将障碍物距离和路径偏差转换为电信号；（3）伺服电机提供振动反馈以进行引导和警报。初步测试证实，该系统能够有效实时检测障碍物并纠正路径偏差，表明其有潜力协助视障用户。然而，由于这是一项正在进行的工作，需要与视障参与者进行全面的实地试验，以充分验证其有效性。
