Mournet Annabelle M, Kleiman Evan M
Department of Psychology, Rutgers, The State University of New Jersey, New Brunswick, New Jersey, USA.
Cyberpsychol Behav Soc Netw. 2023 Feb;26(2):73-79. doi: 10.1089/cyber.2022.0173. Epub 2023 Feb 1.
This study was a multistage process of recruiting participants through Reddit with the intent of increasing data integrity when facing an infiltration of Internet bots. Approaches to increase data integrity centered around preventing the occurrence of Internet bots from the onset and increasing the ability to identify Internet bot responses. We attempted to detect bots in a study focused on understanding social factors related to autism and suicide risk. Four recruitment rounds occurred through Reddit on mental health-related subreddits, with one post made on each subreddit per recruitment round. We found high presence of bots in the initial rounds-indeed, using location data, one third of the total responses (33.4 percent; 118/353) came from just eight locations (i.e., 4.7 percent of all locations). The proportion of detected bots was significantly different across the rounds of recruitment (χ = 150.22, df = 3, < 0.001). In round 4, language advertising compensation was removed from recruitment posts. This round had significantly lower proportions of detected bots compared with round 1 (χ = 33.01, df = 1, < 0.001), round 2 (χ = 129.14, df = 1, < 0.001), and round 3 (χ = 46.6, df = 1, < 0.001). Through a multistage recruitment process, we were able to increase the integrity of our collected data, as determined by a low percentage of fraudulent responses. Only once we removed advertisement of compensation in recruitment posts, did we see a significant decrease in the quantity and percentage of Internet bot responses. This multistage recruitment study provides valuable information regarding how to adapt when an online survey study is infiltrated with Internet bots.
本研究是一个多阶段的过程，通过Reddit招募参与者，目的是在面对互联网机器人渗透时提高数据完整性。提高数据完整性的方法主要围绕从一开始就防止互联网机器人的出现，并增强识别互联网机器人回复的能力。我们试图在一项专注于理解与自闭症和自杀风险相关的社会因素的研究中检测机器人。通过Reddit在与心理健康相关的子版块上进行了四轮招募，每轮招募在每个子版块上发布一篇帖子。我们发现最初几轮中机器人的出现频率很高——实际上，利用位置数据，总回复量的三分之一（33.4%；118/353）仅来自八个位置（即所有位置的4.7%）。在各轮招募中，检测到的机器人比例存在显著差异（χ = 150.22，自由度 = 3， < 0.001）。在第4轮中，招募帖子中不再提及语言广告报酬。与第1轮（χ = 33.01，自由度 = 1， < 0.001）、第2轮（χ = 129.14，自由度 = 1， < 0.001）和第3轮（χ = 46.6，自由度 = 1， < 0.001）相比，这一轮检测到的机器人比例显著降低。通过多阶段招募过程，我们能够提高所收集数据的完整性，这由低比例的欺诈性回复来确定。只有当我们在招募帖子中不再提及报酬广告时，我们才看到互联网机器人回复的数量和比例显著下降。这项多阶段招募研究提供了有关在线调查研究被互联网机器人渗透时如何应对的宝贵信息。
