Craig Leslie S, Evans Christina L, Taylor Brittany D, Patterson Jace, Whitfield Kaleb, Hill Mekhi, Nwagwu Michelle, Mubasher Mohamed, Bednarczyk Robert A, McCray Gail G, Gaddis Cheryl L R, Taylor Natasha, Thompson Emily, Douglas Ursula, Latimer Saundra K, Spivey Sedessie G, Henry Akintobi Tabia, Quarells Rakale Collins
Strategic Consulting Solutions, Christ Church, Barbados.
Department of Family Medicine, Morehouse School of Medicine, Atlanta, GA, United States.
JMIR Public Health Surveill. 2024 Dec 24;10:e51786. doi: 10.2196/51786.
Convenience, privacy, and cost-effectiveness associated with web-based data collection have facilitated the recent expansion of web-based survey research. Importantly, however, practical benefits of web-based survey research, to scientists and participants alike, are being overshadowed by the dramatic rise in suspicious and fraudulent survey submissions. Misinformation associated with survey fraud compromises data quality and data integrity with important implications for scientific conclusions, clinical practice, and social benefit. Transparency in reporting on methods used to prevent and manage suspicious and fraudulent submissions is key to protecting the veracity of web-based survey data; yet, there is limited discussion on the use of antideception strategies during all phases of survey research to detect and eliminate low-quality and fraudulent responses.
This study aims to contribute to an evolving evidence base on data integrity threats associated with web-based survey research by describing study design strategies and antideception tools used during the web-based administration of the Garnering Effective Outreach and Research in Georgia for Impact Alliance-Community Engagement Alliance (GEORGIA CEAL) Against COVID-19 Disparities project surveys.
GEORGIA CEAL was established in response to the COVID-19 pandemic and the need for rapid, yet, valid, community-informed, and community-owned research to guide targeted responses to a dynamic, public health crisis. GEORGIA CEAL Surveys I (April 2021 to June 2021) and II (November 2021 to January 2022) received institutional review board approval from the Morehouse School of Medicine and adhered to the CHERRIES (Checklist for Reporting Results of Internet E-Surveys).
A total of 4934 and 4905 submissions were received for Surveys I and II, respectively. A small proportion of surveys (Survey I: n=1336, 27.1% and Survey II: n=1024, 20.9%) were excluded due to participant ineligibility, while larger proportions (Survey I: n=1516, 42.1%; Survey II: n=1423, 36.7%) were flagged and removed due to suspicious activity; 2082 (42.2%) and 2458 (50.1%) of GEORGIA CEAL Surveys I and II, respectively, were retained for analysis.
Suspicious activity during GEORGIA CEAL Survey I administration prompted the inclusion of additional security tools during Survey II design and administration (eg, hidden questions, Completely Automated Public Turing Test to Tell Computers and Humans Apart verification, and security questions), which proved useful in managing and detecting fraud and resulted in a higher retention rate across survey waves. By thorough discussion of experiences, lessons learned, and future directions for web-based survey research, this study outlines challenges and best practices for designing and implementing a robust defense against survey fraud. Finally, we argue that, in addition to greater transparency and discussion, community stakeholders need to be intentionally and mindfully engaged, via approaches grounded in community-based participatory research, around the potential for research to enable scientific discoveries in order to accelerate investment in quality, legitimate survey data.
基于网络的数据收集所具有的便利性、隐私性和成本效益，推动了近期基于网络的调查研究的扩展。然而，重要的是，基于网络的调查研究给科学家和参与者带来的实际益处，正被可疑及欺诈性调查提交数量的急剧增加所掩盖。与调查欺诈相关的错误信息会损害数据质量和数据完整性，对科学结论、临床实践和社会效益产生重要影响。报告用于预防和管理可疑及欺诈性提交的方法的透明度，是保护基于网络的调查数据准确性的关键；然而，在调查研究的各个阶段使用反欺骗策略以检测和消除低质量及欺诈性回复的讨论却很有限。
本研究旨在通过描述在佐治亚州获取有效外展和研究以促进抗击新冠疫情差异联盟 - 社区参与联盟（GEORGIA CEAL）抗击新冠疫情差异项目调查的网络管理过程中所使用的研究设计策略和反欺骗工具，为基于网络的调查研究相关的数据完整性威胁这一不断发展的证据库做出贡献。
GEORGIA CEAL是为应对新冠疫情以及对快速、有效、基于社区信息且由社区主导的研究的需求而设立的，旨在指导针对动态公共卫生危机的有针对性应对措施。GEORGIA CEAL调查I（2021年4月至2021年6月）和调查II（2021年11月至2022年1月）获得了莫尔豪斯医学院机构审查委员会的批准，并遵循了CHERRIES（互联网电子调查结果报告清单）。
调查I和调查II分别共收到4934份和4905份提交。一小部分调查（调查I：n = 1336，27.1%；调查II：n = 1024，20.9%）因参与者不符合资格而被排除，而更大比例（调查I：n = 1516，42.1%；调查II：n = 1423，36.7%）因可疑活动被标记并移除；GEORGIA CEAL调查I和调查II分别有2082份（42.2%）和2458份（50.1%）被保留用于分析。
GEORGIA CEAL调查I管理期间的可疑活动促使在调查II的设计和管理过程中纳入了额外的安全工具（如隐藏问题、全自动区分计算机和人类的图灵测试验证以及安全问题），这些工具在管理和检测欺诈方面被证明是有用的，并在各轮调查中带来了更高的保留率。通过对基于网络的调查研究的经验、教训和未来方向进行全面讨论，本研究概述了设计和实施强大的调查欺诈防御措施的挑战和最佳实践。最后，我们认为，除了提高透明度和进行更多讨论外，还需要通过基于社区参与式研究的方法，有意且谨慎地让社区利益相关者参与到围绕研究潜力的讨论中，以便实现科学发现，从而加速对高质量、合法调查数据的投入。
