Bhavna Km, Akhter Azman, Banerjee Romi, Roy Dipanjan
Department of Computer Science and Engineering, IIT Jodhpur, Karwar, Rajasthan, India.
Cognitive Brain Dynamics Lab, National Brain Research Centre, Manesar, Gurugram, India.
Front Neuroinform. 2024 Jun 28;18:1392661. doi: 10.3389/fninf.2024.1392661. eCollection 2024.
Decoding of cognitive states aims to identify individuals' brain states and brain fingerprints to predict behavior. Deep learning provides an important platform for analyzing brain signals at different developmental stages to understand brain dynamics. Due to their internal architecture and feature extraction techniques, existing machine-learning and deep-learning approaches are suffering from low classification performance and explainability issues that must be improved. In the current study, we hypothesized that even at the early childhood stage (as early as 3-years), connectivity between brain regions could decode brain states and predict behavioral performance in false-belief tasks. To this end, we proposed an explainable deep learning framework to decode brain states (Theory of Mind and Pain states) and predict individual performance on ToM-related false-belief tasks in a developmental dataset. We proposed an explainable spatiotemporal connectivity-based Graph Convolutional Neural Network (Ex-stGCNN) model for decoding brain states. Here, we consider a developmental dataset, = 155 (122 children; 3-12 yrs and 33 adults; 18-39 yrs), in which participants watched a short, soundless animated movie, shown to activate Theory-of-Mind (ToM) and pain networs. After scanning, the participants underwent a ToM-related false-belief task, leading to categorization into the pass, fail, and inconsistent groups based on performance. We trained our proposed model using Functional Connectivity (FC) and Inter-Subject Functional Correlations (ISFC) matrices separately. We observed that the stimulus-driven feature set (ISFC) could capture ToM and Pain brain states more accurately with an average accuracy of 94%, whereas it achieved 85% accuracy using FC matrices. We also validated our results using five-fold cross-validation and achieved an average accuracy of 92%. Besides this study, we applied the SHapley Additive exPlanations (SHAP) approach to identify brain fingerprints that contributed the most to predictions. We hypothesized that ToM network brain connectivity could predict individual performance on false-belief tasks. We proposed an Explainable Convolutional Variational Auto-Encoder (Ex-Convolutional VAE) model to predict individual performance on false-belief tasks and trained the model using FC and ISFC matrices separately. ISFC matrices again outperformed the FC matrices in prediction of individual performance. We achieved 93.5% accuracy with an F1-score of 0.94 using ISFC matrices and achieved 90% accuracy with an F1-score of 0.91 using FC matrices.
认知状态解码旨在识别个体的脑状态和脑指纹以预测行为。深度学习为分析不同发育阶段的脑信号以理解脑动力学提供了一个重要平台。由于其内部架构和特征提取技术，现有的机器学习和深度学习方法存在分类性能低和可解释性问题，这些问题必须得到改善。在当前研究中，我们假设即使在幼儿期（早在3岁时），脑区之间的连接性也能够解码脑状态并预测错误信念任务中的行为表现。为此，我们提出了一个可解释的深度学习框架，用于在一个发育数据集中解码脑状态（心理理论和疼痛状态）并预测个体在与心理理论相关的错误信念任务中的表现。我们提出了一种基于可解释的时空连接性的图卷积神经网络（Ex-stGCNN）模型来解码脑状态。在此，我们考虑一个发育数据集，N = 155（122名儿童；3至12岁，33名成年人；18至39岁），其中参与者观看了一部无声的短动画电影，该电影被证明能激活心理理论（ToM）和疼痛网络。扫描后，参与者进行了一项与心理理论相关的错误信念任务，根据表现被分类为通过、未通过和不一致组。我们分别使用功能连接（FC）和个体间功能相关性（ISFC）矩阵训练我们提出的模型。我们观察到，刺激驱动的特征集（ISFC）能够更准确地捕捉心理理论和疼痛脑状态，平均准确率为94%，而使用FC矩阵时准确率为85%。我们还使用五折交叉验证对结果进行了验证，平均准确率达到了92%。除此之外，我们应用夏普利值附加解释（SHAP）方法来识别对预测贡献最大的脑指纹。我们假设心理理论网络的脑连接性能够预测错误信念任务中的个体表现。我们提出了一个可解释的卷积变分自编码器（Ex-卷积VAE）模型来预测错误信念任务中的个体表现，并分别使用FC和ISFC矩阵训练该模型。在预测个体表现方面，ISFC矩阵再次优于FC矩阵。使用ISFC矩阵时，我们的准确率达到了93.5%，F1分数为0.94；使用FC矩阵时，准确率为90%，F1分数为0.91。
