Faculty IV - Electrical Engineering and Computer Science, TU Berlin, Berlin, 10587, Germany.
School of Interactive Computing, College of Computing, Georgia Institute of Technology, Atlanta, GA, 30332, USA.
Sci Rep. 2020 Aug 3;10(1):13035. doi: 10.1038/s41598-020-69807-0.
When retrieving image from memory, humans usually move their eyes spontaneously as if the image were in front of them. Such eye movements correlate strongly with the spatial layout of the recalled image content and function as memory cues facilitating the retrieval procedure. However, how close the correlation is between imagery eye movements and the eye movements while looking at the original image is unclear so far. In this work we first quantify the similarity of eye movements between recalling an image and encoding the same image, followed by the investigation on whether comparing such pairs of eye movements can be used for computational image retrieval. Our results show that computational image retrieval based on eye movements during spontaneous imagery is feasible. Furthermore, we show that such a retrieval approach can be generalized to unseen images.
从记忆中检索图像时，人类通常会自发地移动眼睛，就好像图像就在他们面前一样。这种眼动与回忆图像内容的空间布局密切相关，并且作为记忆线索，有助于检索过程。然而，到目前为止，还不清楚想象中的眼动与观看原始图像时的眼动之间的相关性有多强。在这项工作中，我们首先量化了回忆图像和编码同一图像之间眼动的相似性，然后研究了比较这样的眼动对是否可用于计算图像检索。我们的结果表明，基于自发想象时的眼动进行计算图像检索是可行的。此外，我们还表明，这种检索方法可以推广到看不见的图像。
