Balaban Halely, Smith Kevin A, Tenenbaum Joshua B, Ullman Tomer D
Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, USA.
Department of Psychology, Harvard University, Cambridge, MA, USA.
Open Mind (Camb). 2024 Nov 22;8:1425-1446. doi: 10.1162/opmi_a_00174. eCollection 2024.
Starting in early infancy, our perception and predictions are rooted in strong expectations about the behavior of everyday objects. These intuitive physics expectations have been demonstrated in numerous behavioral experiments, showing that even pre-verbal infants are surprised when something impossible happens (e.g., when objects magically appear or disappear). However, it remains unclear whether and how physical expectations shape different aspects of moment-by-moment online visual scene processing, unrelated to explicit physical reasoning. In two EEG experiments, people watched short videos like those used in behavioral studies with adults and infants, and more recently in AI benchmarks. Objects moved on a stage, and were briefly hidden behind an occluder, with the scene either unfolding as expected, or violating object permanence (adding or removing an object). We measured the contralateral delay activity, an electrophysiological marker of online processing, to examine participants' working memory (WM) representations, as well as their ability to continuously track the objects in the scene. We found that both types of object permanence violations disrupted tracking, even though violations involved perceptually non-salient events (magical vanishing) or new objects that weren't previously tracked (magical creation). Physical violations caused WM to reset, i.e., to discard the original scene representation before it could recover and represent the updated number of items. Providing a physical explanation for the violations (a hole behind the occluder) restored object tracking, and we found evidence that WM continued to represent items that disappeared 'down the hole'. Our results show how intuitive physical expectations shape online representations, and form the basis of dynamic object tracking.
从婴儿早期开始,我们的感知和预测就基于对日常物体行为的强烈预期。这些直观的物理预期已在众多行为实验中得到证明,表明即使是还不会说话的婴儿,当出现不可能的事情时(例如,物体神奇地出现或消失)也会感到惊讶。然而,尚不清楚物理预期是否以及如何塑造即时在线视觉场景处理的不同方面,这与明确的物理推理无关。在两项脑电图实验中,人们观看了与成人和婴儿行为研究中使用的类似短视频,以及最近人工智能基准测试中使用的短视频。物体在舞台上移动,并短暂地隐藏在遮挡物后面,场景要么按预期展开,要么违反物体恒存性(添加或移除一个物体)。我们测量了对侧延迟活动,这是一种在线处理的电生理指标,以检查参与者的工作记忆(WM)表征,以及他们连续跟踪场景中物体的能力。我们发现,两种类型的物体恒存性违反都会干扰跟踪,即使这些违反涉及感知上不突出的事件(神奇消失)或之前未被跟踪的新物体(神奇出现)。物理违反会导致工作记忆重置,即在其恢复并表征更新后的物品数量之前丢弃原始场景表征。为这些违反情况提供物理解释(遮挡物后面有一个洞)可恢复物体跟踪,并且我们发现有证据表明工作记忆继续表征从“洞中”消失的物品。我们的结果表明直观的物理预期如何塑造在线表征,并构成动态物体跟踪的基础。