Graduate Program in Integrative Neuroscience, University of Nevada, Reno, USA.
Department of Psychology, University of Nevada, Reno, USA.
Behav Res Methods. 2024 Apr;56(4):3959-3981. doi: 10.3758/s13428-023-02266-3. Epub 2023 Nov 28.
Deep neural networks (DNNs) have enabled recent advances in the accuracy and robustness of video-oculography. However, to make robust predictions, most DNN models require extensive and diverse training data, which is costly to collect and label. In this work, we seek to improve the codevelop pylids, a pupil- and eyelid-estimation DNN model based on DeepLabCut. We show that performance of pylids-based pupil estimation can be related to the distance of test data from the distribution of training data. Based on this principle, we explore methods for efficient data selection for training our DNN. We show that guided sampling of new data points from the training data approaches state-of-the-art pupil and eyelid estimation with fewer training data points. We also demonstrate the benefit of using an efficient sampling method to select data augmentations for training DNNs. These sampling methods aim to minimize the time and effort required to label and train DNNs while promoting model generalization on new diverse datasets.
深度神经网络（DNNs）已经使得视频眼动追踪的准确性和稳健性得到了最近的进展。然而，为了做出稳健的预测，大多数 DNN 模型都需要广泛和多样化的训练数据，这在收集和标记方面是昂贵的。在这项工作中，我们旨在改进基于 DeepLabCut 的瞳孔和眼睑估计 DNN 模型 pylids。我们表明，基于 pylids 的瞳孔估计性能可以与测试数据与训练数据分布的距离相关。基于这一原理，我们探索了用于训练我们的 DNN 的有效数据选择方法。我们表明，从训练数据中引导性地选择新数据点可以用更少的训练数据点接近最先进的瞳孔和眼睑估计。我们还展示了使用高效的采样方法选择数据增强进行 DNN 训练的好处。这些采样方法旨在在促进新的多样化数据集上的模型泛化的同时，最小化标记和训练 DNN 所需的时间和精力。
