Luck Steven J, Gaspelin Nicholas
Center for Mind & Brain, University of California, Davis, Davis, California, USA.
Department of Psychology, University of California, Davis, Davis, California, USA.
Psychophysiology. 2017 Jan;54(1):146-157. doi: 10.1111/psyp.12639.
ERP experiments generate massive datasets, often containing thousands of values for each participant, even after averaging. The richness of these datasets can be very useful in testing sophisticated hypotheses, but this richness also creates many opportunities to obtain effects that are statistically significant but do not reflect true differences among groups or conditions (bogus effects). The purpose of this paper is to demonstrate how common and seemingly innocuous methods for quantifying and analyzing ERP effects can lead to very high rates of significant but bogus effects, with the likelihood of obtaining at least one such bogus effect exceeding 50% in many experiments. We focus on two specific problems: using the grand-averaged data to select the time windows and electrode sites for quantifying component amplitudes and latencies, and using one or more multifactor statistical analyses. Reanalyses of prior data and simulations of typical experimental designs are used to show how these problems can greatly increase the likelihood of significant but bogus results. Several strategies are described for avoiding these problems and for increasing the likelihood that significant effects actually reflect true differences among groups or conditions.
事件相关电位（ERP）实验会生成海量数据集，即使经过平均处理，每个参与者通常仍包含数千个数据值。这些数据集的丰富性在检验复杂假设时非常有用，但这种丰富性也带来了许多机会，使得获得的效应在统计上具有显著性，但并不反映组间或条件间的真实差异（虚假效应）。本文的目的是说明，用于量化和分析ERP效应的常见且看似无害的方法如何导致出现大量显著但虚假的效应，在许多实验中获得至少一个此类虚假效应的可能性超过50%。我们关注两个具体问题：使用总体平均数据来选择用于量化成分波幅和潜伏期的时间窗口和电极位点，以及使用一种或多种多因素统计分析方法。通过对先前数据的重新分析和对典型实验设计的模拟，来展示这些问题如何能极大地增加出现显著但虚假结果的可能性。文中描述了几种策略，用于避免这些问题，并提高显著效应实际反映组间或条件间真实差异的可能性。
