Brysbaert Marc, Martínez Gonzalo, Reviriego Pedro
Department of Experimental Psychology, Ghent University, 9000, Ghent, Belgium.
Universidad Carlos III de Madrid, Avenida de la Universidad, 30, 28911, Leganés, Madrid, Spain.
Behav Res Methods. 2024 Dec 28;57(1):28. doi: 10.3758/s13428-024-02561-7.
This study investigates the potential of large language models (LLMs) to estimate the familiarity of words and multi-word expressions (MWEs). We validated LLM estimates for isolated words using existing human familiarity ratings and found strong correlations. LLM familiarity estimates performed even better in predicting lexical decision and naming performance in megastudies than the best available word frequency measures. We then applied LLM estimates to MWEs, also finding their effectiveness in measuring familiarity for these expressions. We have created a list of more than 400,000 English words and MWEs with LLM-generated familiarity estimates, which we hope will be a valuable resource for researchers. There is also a cleaned-up list of nearly 150,000 entries, excluding lesser-known stimuli, to streamline stimulus selection. Our findings highlight the advantages of LLM-based familiarity estimates, including their better performance than traditional word frequency measures (particularly for predicting word recognition accuracy), their ability to generalize to MWEs, availability for large lists of words, and ease of obtaining new estimates for all types of stimuli.
本研究调查了大语言模型（LLMs）估计单词和多词表达式（MWEs）熟悉度的潜力。我们使用现有的人类熟悉度评分验证了大语言模型对孤立单词的估计，并发现了很强的相关性。在大型研究中，大语言模型的熟悉度估计在预测词汇判断和命名表现方面比现有的最佳词频指标表现更好。然后，我们将大语言模型的估计应用于多词表达式，也发现它们在测量这些表达式的熟悉度方面是有效的。我们创建了一个包含超过40万个英语单词和多词表达式的列表，并给出了由大语言模型生成的熟悉度估计，我们希望这将成为研究人员的宝贵资源。还有一个经过清理的列表，包含近15万个条目，排除了不太知名的刺激词，以简化刺激词的选择。我们的研究结果突出了基于大语言模型的熟悉度估计的优势，包括它们比传统词频指标表现更好（特别是在预测单词识别准确性方面），能够推广到多词表达式，可用于大量单词列表，以及易于获得所有类型刺激词的新估计。
