National key Laboratory of Human Machine and Environment Engineering, School of Aeronautical Science and Engineering, Beihang University, Beijing 100191, China.
Biosensors (Basel). 2022 Jun 12;12(6):404. doi: 10.3390/bios12060404.
With the development of the autopilot system, the main task of a pilot has changed from controlling the aircraft to supervising the autopilot system and making critical decisions. Therefore, the human-machine interaction system needs to be improved accordingly. A key step to improving the human-machine interaction system is to improve its understanding of the pilots' status, including fatigue, stress, workload, etc. Monitoring pilots' status can effectively prevent human error and achieve optimal human-machine collaboration. As such, there is a need to recognize pilots' status and predict the behaviors responsible for changes of state. For this purpose, in this study, 14 Air Force cadets fly in an F-35 Lightning II Joint Strike Fighter simulator through a series of maneuvers involving takeoff, level flight, turn and hover, roll, somersault, and stall. Electro cardio (ECG), myoelectricity (EMG), galvanic skin response (GSR), respiration (RESP), and skin temperature (SKT) measurements are derived through wearable physiological data collection devices. Physiological indicators influenced by the pilot's behavioral status are objectively analyzed. Multi-modality fusion technology (MTF) is adopted to fuse these data in the feature layer. Additionally, four classifiers are integrated to identify pilots' behaviors in the strategy layer. The results indicate that MTF can help to recognize pilot behavior in a more comprehensive and precise way.
随着自动驾驶系统的发展，飞行员的主要任务已经从控制飞机转变为监督自动驾驶系统和做出关键决策。因此，人机交互系统需要相应地改进。改进人机交互系统的关键步骤是提高其对飞行员状态的理解，包括疲劳、压力、工作量等。监测飞行员的状态可以有效防止人为错误，实现最佳人机协作。因此，需要识别飞行员的状态并预测导致状态变化的行为。为此，在这项研究中，14 名空军学员在 F-35 闪电 II 联合攻击战斗机模拟器中通过一系列机动飞行，包括起飞、水平飞行、转弯和悬停、滚转、筋斗和失速。通过可穿戴生理数据采集设备得出心电（ECG）、肌电（EMG）、皮肤电反应（GSR）、呼吸（RESP）和皮肤温度（SKT）测量值。客观分析受飞行员行为状态影响的生理指标。采用多模态融合技术（MTF）在特征层融合这些数据。此外，在策略层集成了四个分类器来识别飞行员的行为。结果表明，MTF 可以帮助更全面、更精确地识别飞行员的行为。
