Medical Imaging Center, Department of Electronic Engineering and Information Science, University of Science and Technology of China, Hefei, People's Republic of China.
Institute of Artificial Intelligence, Hefei Comprehensive National Science Center, Hefei, People's Republic of China.
J Neural Eng. 2024 Aug 13;21(4). doi: 10.1088/1741-2552/ad6185.
. Eye-tracking research has proven valuable in understanding numerous cognitive functions. Recently, Freyprovided an exciting deep learning method for learning eye movements from functional magnetic resonance imaging (fMRI) data. It employed the multi-step co-registration of fMRI into the group template to obtain eyeball signal, and thus required additional templates and was time consuming. To resolve this issue, in this paper, we propose a framework named MRGazer for predicting eye gaze points from fMRI in individual space.. The MRGazer consists of an eyeball extraction module and a residual network-based eye gaze prediction module. Compared to the previous method, the proposed framework skips the fMRI co-registration step, simplifies the processing protocol, and achieves end-to-end eye gaze regression.. The proposed method achieved superior performance in eye fixation regression (Euclidean error, EE = 2.04°) than the co-registration-based method (EE = 2.89°), and delivered objective results within a shorter time (∼0.02 s volume) than prior method (∼0.3 s volume).. The MRGazer is an efficient, simple, and accurate deep learning framework for predicting eye movement from fMRI data, and can be employed during fMRI scans in psychological and cognitive research. The code is available athttps://github.com/ustc-bmec/MRGazer.
眼动追踪研究已被证明在理解许多认知功能方面具有重要价值。最近，Frey 提供了一种令人兴奋的深度学习方法，用于从功能磁共振成像 (fMRI) 数据中学习眼球运动。它采用 fMRI 到组模板的多步骤配准来获取眼球信号，因此需要额外的模板并且耗时。为了解决这个问题，在本文中，我们提出了一个名为 MRGazer 的框架，用于在个体空间中从 fMRI 预测眼点。MRGazer 由眼球提取模块和基于残差网络的眼动预测模块组成。与以前的方法相比，该框架跳过了 fMRI 配准步骤，简化了处理协议，并实现了端到端的眼动回归。该方法在眼固定回归方面的性能优于基于配准的方法（欧几里得误差，EE = 2.04°），并且比以前的方法（∼0.3 秒体积）更快地提供了客观结果（∼0.02 秒体积）。MRGazer 是一种用于从 fMRI 数据预测眼动的高效、简单和准确的深度学习框架，可在心理和认知研究中的 fMRI 扫描期间使用。代码可在 https://github.com/ustc-bmec/MRGazer 获得。
