Department of Psychology, University of Utah, 380 S 1530 E BEHS 1003, Salt Lake City, UT, 84112, USA.
Red Scientific Inc., Salt Lake City, UT, USA.
Cogn Res Princ Implic. 2024 Sep 11;9(1):60. doi: 10.1186/s41235-024-00591-5.
The reliability of cognitive demand measures in controlled laboratory settings is well-documented; however, limited research has directly established their stability under real-life and high-stakes conditions, such as operating automated technology on actual highways. Partially automated vehicles have advanced to become an everyday mode of transportation, and research on driving these advanced vehicles requires reliable tools for evaluating the cognitive demand on motorists to sustain optimal engagement in the driving process. This study examined the reliability of five cognitive demand measures, while participants operated partially automated vehicles on real roads across four occasions. Seventy-one participants (aged 18-64 years) drove on actual highways while their heart rate, heart rate variability, electroencephalogram (EEG) alpha power, and behavioral performance on the Detection Response Task were measured simultaneously. Findings revealed that EEG alpha power had excellent test-retest reliability, heart rate and its variability were good, and Detection Response Task reaction time and hit-rate had moderate reliabilities. Thus, the current study addresses concerns regarding the reliability of these measures in assessing cognitive demand in real-world automation research, as acceptable test-retest reliabilities were found across all measures for drivers across occasions. Despite the high reliability of each measure, low intercorrelations among measures were observed, and internal consistency was better when cognitive demand was estimated as a multi-factorial construct. This suggests that they tap into different aspects of cognitive demand while operating automation in real life. The findings highlight that a combination of psychophysiological and behavioral methods can reliably capture multi-faceted cognitive demand in real-world automation research.
认知需求测量在控制实验室环境中的可靠性已有充分记录；然而，只有有限的研究直接确定了它们在现实生活和高风险条件下的稳定性，例如在实际高速公路上操作自动化技术。部分自动化车辆已经发展成为一种日常交通方式，研究驾驶这些先进车辆需要可靠的工具来评估驾驶员在驾驶过程中的认知需求，以保持最佳参与度。本研究在四个不同场合共 71 名参与者（年龄 18-64 岁）在实际道路上操作部分自动化车辆时，考察了五种认知需求测量的可靠性。同时测量了参与者的心率、心率变异性、脑电图（EEG）阿尔法功率和检测反应任务的行为表现。研究结果表明，EEG 阿尔法功率具有极好的重测信度，心率及其变异性良好，检测反应任务的反应时间和击中率具有中等可靠性。因此，本研究解决了在现实世界自动化研究中评估认知需求时这些测量的可靠性问题，因为在所有测量中，在所有情况下，驾驶员都发现了可接受的重测信度。尽管每种测量方法的可靠性都很高，但观察到各测量方法之间的低相关性，并且当将认知需求估计为多因素结构时，内部一致性更好。这表明它们在现实生活中操作自动化时涉及到认知需求的不同方面。研究结果强调，在真实世界的自动化研究中，结合使用生理心理学和行为方法可以可靠地捕捉多方面的认知需求。
