Department of Business and Management, Southern Denmark University, Universitetsparken 1, Kolding, 6000, Denmark.
Behav Res Methods. 2024 Aug;56(5):4823-4835. doi: 10.3758/s13428-023-02220-3. Epub 2023 Sep 1.
One of the issues that can potentially affect the internal validity of interactive online experiments that recruit participants using crowdsourcing platforms is collusion: participants could act upon information shared through channels that are external to the experimental design. Using two experiments, I measure how prevalent collusion is among MTurk workers and whether collusion depends on experimental design choices. Despite having incentives to collude, I find no evidence that MTurk workers collude in the treatments that resembled the design of most other interactive online experiments. This suggests collusion is not a concern for data quality in typical interactive online experiments that recruit participants using crowdsourcing platforms. However, I find that approximately 3% of MTurk workers collude when the payoff of collusion is unusually high. Therefore, collusion should not be overlooked as a possible danger to data validity in interactive experiments that recruit participants using crowdsourcing platforms when participants have strong incentives to engage in such behavior.
参与者可能会根据实验设计之外的渠道共享的信息采取行动。通过两项实验,我衡量了 MTurk 工人中串通的普遍程度,以及串通是否取决于实验设计选择。尽管有串通的动机,但我没有发现 MTurk 工人在类似于大多数其他交互式在线实验设计的处理中串通的证据。这表明,在使用众包平台招募参与者的典型交互式在线实验中,串通不会影响数据质量。然而,当串通的回报异常高时,我发现大约 3%的 MTurk 工人会串通。因此,当参与者有强烈的串通动机时,串通不应该被忽视为使用众包平台招募参与者的交互式实验中数据有效性的一个潜在危险。