Department of Psychology, University of Pittsburgh, Pittsburgh, PA, 15260, USA,
Behav Res Methods. 2014 Mar;46(1):95-111. doi: 10.3758/s13428-013-0345-y.
Crowdsourcing systems like Amazon's Mechanical Turk (AMT) allow data to be collected from a large sample of people in a short amount of time. This use has garnered considerable interest from behavioral scientists. So far, most experiments conducted on AMT have focused on survey-type instruments because of difficulties inherent in running many experimental paradigms over the Internet. This study investigated the viability of presenting stimuli and collecting response times using Adobe Flash to run ActionScript 3 code in conjunction with AMT. First, the timing properties of Adobe Flash were investigated using a phototransistor and two desktop computers running under several conditions mimicking those that may be present in research using AMT. This experiment revealed some strengths and weaknesses of the timing capabilities of this method. Next, a flanker task and a lexical decision task implemented in Adobe Flash were administered to participants recruited with AMT. The expected effects in these tasks were replicated. Power analyses were conducted to describe the number of participants needed to replicate these effects. A questionnaire was used to investigate previously undescribed computer use habits of 100 participants on AMT. We conclude that a Flash program in conjunction with AMT can be successfully used for running many experimental paradigms that rely on response times, although experimenters must understand the limitations of the method.
众包系统，如亚马逊的 Mechanical Turk (AMT)，允许在短时间内从大量人群中收集数据。这种用法引起了行为科学家的极大兴趣。到目前为止，AMT 上进行的大多数实验都集中在调查型工具上，因为在互联网上运行许多实验范式存在固有困难。本研究调查了使用 Adobe Flash 呈现刺激和收集响应时间并结合 AMT 运行 ActionScript 3 代码的可行性。首先，使用光电晶体管和两台在模仿 AMT 研究中可能存在的条件下运行的台式计算机研究了 Adobe Flash 的计时特性。该实验揭示了这种方法计时功能的一些优缺点。接下来，使用 AMT 招募的参与者进行了 Adobe Flash 中的侧抑制任务和词汇判断任务。这些任务中的预期效果得到了复制。进行了功效分析，以描述复制这些效果所需的参与者数量。使用问卷调查了 AMT 上 100 名参与者以前未描述过的计算机使用习惯。我们得出结论，结合 AMT 的 Flash 程序可以成功用于运行许多依赖于响应时间的实验范式，尽管实验者必须了解该方法的局限性。
