Arnold Derek H, Johnston Alan, Adie Joshua, Yarrow Kielan
School of Psychology, The University of Queensland, Australia.
School of Psychology, The University of Nottingham, United Kingdom.
Conscious Cogn. 2023 Aug;113:103532. doi: 10.1016/j.concog.2023.103532. Epub 2023 Jun 7.
Signal-detection theory (SDT) is one of the most popular frameworks for analyzing data from studies of human behavior - including investigations of confidence. SDT-based analyses of confidence deliver both standard estimates of sensitivity (d'), and a second estimate informed by high-confidence decisions - meta d'. The extent to which meta d' estimates fall short of d' estimates is regarded as a measure of metacognitive inefficiency, quantifying the contamination of confidence by additional noise. These analyses rely on a key but questionable assumption - that repeated exposures to an input will evoke a normally-shaped distribution of perceptual experiences (the normality assumption). Here we show, via analyses inspired by an experiment and modelling, that when distributions of experience do not conform with the normality assumption, meta d' can be systematically underestimated relative to d'. Our data highlight that SDT-based analyses of confidence do not provide a ground truth measure of human metacognitive inefficiency. We explain why deviance from the normality assumption is especially a problem for some popular SDT-based analyses of confidence, in contrast to other analyses inspired by the SDT framework, which are more robust to violations of the normality assumption.
信号检测理论（SDT）是分析人类行为研究数据（包括对信心的调查）最常用的框架之一。基于SDT的信心分析既能得出敏感性的标准估计值（d'），还能得出基于高信心决策的第二个估计值——元d'。元d'估计值低于d'估计值的程度被视为元认知效率低下的一种度量，量化了额外噪声对信心的干扰。这些分析依赖于一个关键但有问题的假设——即对某个输入的反复接触会引发感知体验的正态分布（正态性假设）。在此，我们通过受一项实验启发的分析和建模表明，当体验分布不符合正态性假设时，相对于d'，元d'可能会被系统性低估。我们的数据突出表明，基于SDT的信心分析并不能提供人类元认知效率低下的真实度量。我们解释了为什么与受SDT框架启发的其他分析相比，偏离正态性假设对一些基于SDT的流行信心分析尤其成问题，而其他分析对违反正态性假设更具稳健性。
