Pinzón Natalia, Koundinya Vikram, Galt Ryan E, Dowling William O'R, Baukloh Marcela, Taku-Forchu Namah C, Schohr Tracy, Roche Leslie M, Ikendi Samuel, Cooper Mark, Parker Lauren E, Pathak Tapan B
Geography Graduate Group, University of California, Davis, Davis, CA, United States.
Rhizobia LLC, San Francisco, CA, United States.
Front Res Metr Anal. 2024 Dec 2;9:1432774. doi: 10.3389/frma.2024.1432774. eCollection 2024.
The proliferation of AI-powered bots and sophisticated fraudsters poses a significant threat to the integrity of scientific studies reliant on online surveys across diverse disciplines, including health, social, environmental and political sciences. We found a substantial decline in usable responses from online surveys from 75 to 10% in recent years due to survey fraud. Monetary incentives attract sophisticated fraudsters capable of mimicking genuine open-ended responses and verifying information submitted months prior, showcasing the advanced capabilities of online survey fraud today. This study evaluates the efficacy of 31 fraud indicators and six ensembles using two agriculture surveys in California. To evaluate the performance of each indicator, we use predictive power and recall. Predictive power is a novel variation of precision introduced in this study, and both are simple metrics that allow for non-academic survey practitioners to replicate our methods. The best indicators included a novel email address score, MinFraud Risk Score, consecutive submissions, opting-out of incentives, improbable location.
人工智能驱动的机器人和老练的欺诈者的激增，对包括健康、社会、环境和政治科学在内的不同学科中依赖在线调查的科学研究的完整性构成了重大威胁。我们发现，由于调查欺诈，近年来在线调查中可用回复率从75%大幅下降至10%。金钱激励吸引了能够模仿真实开放式回复并核实数月前提交信息的老练欺诈者，展示了当今在线调查欺诈的先进能力。本研究使用加利福尼亚州的两项农业调查评估了31个欺诈指标和六种集成方法的有效性。为了评估每个指标的性能，我们使用预测能力和召回率。预测能力是本研究中引入的一种新颖的精确率变体，两者都是简单的指标，使非学术调查从业者能够复制我们的方法。最佳指标包括一个新颖的电子邮件地址评分、MinFraud风险评分、连续提交、拒绝激励、不合理的位置。
