Department of Education, Vrije Universiteit Amsterdam, and LEARN! Research Institute, Amsterdam, The Netherlands.
Department of Experimental and Applied Psychology, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands.
PLoS Comput Biol. 2024 Sep 25;20(9):e1012117. doi: 10.1371/journal.pcbi.1012117. eCollection 2024 Sep.
Although word predictability is commonly considered an important factor in reading, sophisticated accounts of predictability in theories of reading are lacking. Computational models of reading traditionally use cloze norming as a proxy of word predictability, but what cloze norms precisely capture remains unclear. This study investigates whether large language models (LLMs) can fill this gap. Contextual predictions are implemented via a novel parallel-graded mechanism, where all predicted words at a given position are pre-activated as a function of contextual certainty, which varies dynamically as text processing unfolds. Through reading simulations with OB1-reader, a cognitive model of word recognition and eye-movement control in reading, we compare the model's fit to eye-movement data when using predictability values derived from a cloze task against those derived from LLMs (GPT-2 and LLaMA). Root Mean Square Error between simulated and human eye movements indicates that LLM predictability provides a better fit than cloze. This is the first study to use LLMs to augment a cognitive model of reading with higher-order language processing while proposing a mechanism on the interplay between word predictability and eye movements.
虽然词可预测性通常被认为是阅读的一个重要因素，但阅读理论中对可预测性的复杂解释却很缺乏。阅读的计算模型传统上使用 cloze 规范作为词可预测性的代理，但 cloze 规范究竟能捕捉到什么仍然不清楚。本研究探讨了大型语言模型（LLM）是否可以填补这一空白。通过一种新的并行分级机制实现上下文预测，在给定位置的所有预测词都预先激活，作为上下文确定性的函数，上下文确定性随文本处理的展开而动态变化。通过使用 OB1-reader（一种阅读中单词识别和眼动控制的认知模型）进行阅读模拟，我们比较了当使用 cloze 任务得出的可预测性值与从 LLM（GPT-2 和 LLaMA）得出的值来拟合眼动数据时，模型的拟合情况。模拟和人类眼球运动之间的均方根误差表明，LLM 的可预测性提供了比 cloze 更好的拟合。这是第一个使用 LLM 增强阅读认知模型的高阶语言处理的研究，并提出了一个关于词可预测性和眼球运动之间相互作用的机制。
