Naik Ravi, Rubio-Solis Adrian, Jin Kaizhe, Mylonas George
Hamlyn Centre for Robotic Surgery, Imperial College London, London, SW7 2AZ, UK; Department of Surgery and Cancer, St Mary's Hospital, Imperial College London, London, UK.
Eur J Surg Oncol. 2025 Jul;51(7):108735. doi: 10.1016/j.ejso.2024.108735. Epub 2024 Oct 15.
Surgeons can experience elevated cognitive workload (CWL) during surgery due to various factors including operative technicalities and the environmental demands of the operating theatre. This can result in poorer outcomes and have a detrimental effect on surgeon well-being. The objective measurement of CWL provides a potential solution to facilitate classification of workload levels, however results are variable when physiological measures are used in isolation. The aim of this study is to develop and propose a multimodal machine learning (ML) approach to classify CWL levels using a bespoke sensor platform and to develop a ML approach to impute missing pupil diameter measures due to the effect of blinking or noise.
Ten surgical trainees performed a simulated laparoscopic cholecystectomy under cognitive conditions of increasing difficulty, namely a modified auditory N-back task with increasing difficulty and a verbal clinical scenario. Physiological measures were recorded using a novel platform (MAESTRO). Electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) were used as direct measures of CWL. Indirect measures included electromyography (EMG), electrocardiography (ECG) and pupil diameter (PD). A reference point for validation was provided by subjective assessment of perceived CWL using the SURG-TLX. A multimodal machine learning approach that systematically implements a CNN-BiLSTM, a binary version of the metaheuristic Manta Ray Foraging Optimisation (BMRFO) and a version of Fuzzy C-Means (FCM) called Optimal Completion Strategy (OCS) was used to classify the associated perceived CWL state.
Compared to other state of the art classification techniques, cross-validation results for the classification of CWL levels suggest that the CNN-BLSTM and BMRFO approach provides an average accuracy of 97 % based on the confusion matrix. Additionally, OCS demonstrated a superior average performance of 9.15 % in terms of Root-Mean-Square-Error (RMSE) when compared to other PD imputation methods.
Perceived CWL levels were correctly classified using a multimodal ML approach. This approach provides a potential route to accurately classify CWL levels, which may have application in future surgical training and assessment programs as well as the development of cognitive support systems in the operating room.
由于包括手术技术和手术室环境需求等各种因素，外科医生在手术过程中可能会经历认知工作量（CWL）升高的情况。这可能导致较差的手术结果，并对外科医生的身心健康产生不利影响。CWL的客观测量为促进工作量水平分类提供了一种潜在的解决方案，然而，单独使用生理测量时结果存在差异。本研究的目的是开发并提出一种多模态机器学习（ML）方法，使用定制传感器平台对CWL水平进行分类，并开发一种ML方法来估算由于眨眼或噪声影响而缺失的瞳孔直径测量值。
十名外科实习生在认知难度逐渐增加的条件下进行了模拟腹腔镜胆囊切除术，即难度逐渐增加的改良听觉N-回溯任务和言语临床场景。使用新型平台（MAESTRO）记录生理测量值。脑电图（EEG）和功能性近红外光谱（fNIRS）被用作CWL的直接测量指标。间接测量指标包括肌电图（EMG）、心电图（ECG）和瞳孔直径（PD）。使用SURG-TLX对感知到的CWL进行主观评估，提供验证的参考点。一种多模态机器学习方法被用于对相关的感知CWL状态进行分类，该方法系统地实现了卷积神经网络-双向长短期记忆网络（CNN-BiLSTM）、元启发式蝠鲼觅食优化算法（Manta Ray Foraging Optimisation，MRFO）的二元版本（BMRFO）以及一种称为最优完成策略（Optimal Completion Strategy，OCS）的模糊C均值（Fuzzy C-Means，FCM）版本。
与其他现有分类技术相比，CWL水平分类的交叉验证结果表明，基于混淆矩阵，CNN-BLSTM和BMRFO方法的平均准确率为97%。此外，与其他瞳孔直径估算方法相比，OCS在均方根误差（RMSE）方面表现出9.15%的卓越平均性能。
使用多模态ML方法能够正确分类感知到的CWL水平。这种方法为准确分类CWL水平提供了一条潜在途径，可能在未来的外科手术培训和评估项目以及手术室认知支持系统的开发中得到应用。
