Phyllis F Cantor Center for Research in Nursing and Patient Care Services, Dana-Farber Cancer Institute, Boston, MA, United States.
School of Nursing, University of Rochester, Rochester, NY, United States.
J Med Internet Res. 2020 Oct 7;22(10):e23021. doi: 10.2196/23021.
Recruitment of health research participants through social media is becoming more common. In the United States, 80% of adults use at least one social media platform. Social media platforms may allow researchers to reach potential participants efficiently. However, online research methods may be associated with unique threats to sample validity and data integrity. Limited research has described issues of data quality and authenticity associated with the recruitment of health research participants through social media, and sources of low-quality and fraudulent data in this context are poorly understood.
The goal of the research was to describe and explain threats to sample validity and data integrity following recruitment of health research participants through social media and summarize recommended strategies to mitigate these threats. Our experience designing and implementing a research study using social media recruitment and online data collection serves as a case study.
Using published strategies to preserve data integrity, we recruited participants to complete an online survey through the social media platforms Twitter and Facebook. Participants were to receive $15 upon survey completion. Prior to manually issuing remuneration, we reviewed completed surveys for indicators of fraudulent or low-quality data. Indicators attributable to respondent error were labeled suspicious, while those suggesting misrepresentation were labeled fraudulent. We planned to remove cases with 1 fraudulent indicator or at least 3 suspicious indicators.
Within 7 hours of survey activation, we received 271 completed surveys. We classified 94.5% (256/271) of cases as fraudulent and 5.5% (15/271) as suspicious. In total, 86.7% (235/271) provided inconsistent responses to verifiable items and 16.2% (44/271) exhibited evidence of bot automation. Of the fraudulent cases, 53.9% (138/256) provided a duplicate or unusual response to one or more open-ended items and 52.0% (133/256) exhibited evidence of inattention.
Research findings from several disciplines suggest studies in which research participants are recruited through social media are susceptible to data quality issues. Opportunistic individuals who use virtual private servers to fraudulently complete research surveys for profit may contribute to low-quality data. Strategies to preserve data integrity following research participant recruitment through social media are limited. Development and testing of novel strategies to prevent and detect fraud is a research priority.
通过社交媒体招募健康研究参与者变得越来越普遍。在美国，80%的成年人至少使用一个社交媒体平台。社交媒体平台可以让研究人员有效地接触到潜在的参与者。然而，在线研究方法可能与样本有效性和数据完整性的独特威胁有关。有限的研究描述了通过社交媒体招募健康研究参与者相关的数据质量和真实性问题，以及这种情况下低质量和欺诈数据的来源理解得很差。
本研究的目的是描述和解释通过社交媒体招募健康研究参与者后对样本有效性和数据完整性的威胁，并总结减轻这些威胁的建议策略。我们设计和实施一项使用社交媒体招募和在线数据收集的研究的经验作为案例研究。
使用已发布的策略来维护数据完整性，我们通过社交媒体平台 Twitter 和 Facebook 招募参与者完成在线调查。参与者完成调查后将获得 15 美元的报酬。在手动发放报酬之前，我们审查了已完成的调查，以寻找欺诈或低质量数据的指标。归因于受访者错误的指标被标记为可疑，而那些表明有代表性的指标被标记为欺诈。我们计划删除有 1 个欺诈指标或至少 3 个可疑指标的案例。
在调查启动后的 7 个小时内，我们收到了 271 份完成的调查。我们将 94.5%（256/271）的案例归类为欺诈，5.5%（15/271）的案例归类为可疑。总共，86.7%（235/271）的案例对可验证项目的回答不一致，16.2%（44/271）表现出自动化机器人的证据。在欺诈案例中，53.9%（138/256）对一个或多个开放式项目的回答是重复的或不寻常的，52.0%（133/256）表现出注意力不集中的证据。
来自多个学科的研究结果表明，通过社交媒体招募研究参与者的研究容易受到数据质量问题的影响。利用虚拟专用服务器为了盈利而欺诈性地完成研究调查的机会主义者可能会导致低质量的数据。社交媒体招募研究参与者后维护数据完整性的策略有限。开发和测试预防和检测欺诈的新策略是一个研究重点。
