McAndrew Thomas C, Guseva Elizaveta A, Bagrow James P
Mathematics & Statistics, Vermont Complex Systems Center, University of Vermont, Burlington, Vermont, United States of America.
PLoS One. 2017 Aug 14;12(8):e0182662. doi: 10.1371/journal.pone.0182662. eCollection 2017.
Crowdsourcing works by distributing many small tasks to large numbers of workers, yet the true potential of crowdsourcing lies in workers doing more than performing simple tasks-they can apply their experience and creativity to provide new and unexpected information to the crowdsourcer. One such case is when workers not only answer a crowdsourcer's questions but also contribute new questions for subsequent crowd analysis, leading to a growing set of questions. This growth creates an inherent bias for early questions since a question introduced earlier by a worker can be answered by more subsequent workers than a question introduced later. Here we study how to perform efficient crowdsourcing with such growing question sets. By modeling question sets as networks of interrelated questions, we introduce algorithms to help curtail the growth bias by efficiently distributing workers between exploring new questions and addressing current questions. Experiments and simulations demonstrate that these algorithms can efficiently explore an unbounded set of questions without losing confidence in crowd answers.
众包通过将许多小任务分配给大量工人来运作，然而众包的真正潜力在于工人不仅仅是执行简单任务——他们可以运用自己的经验和创造力为众包者提供新的、意想不到的信息。一个这样的例子是，工人不仅回答众包者的问题，还为后续的群体分析贡献新问题，从而导致问题集不断增长。这种增长对早期问题产生了一种内在的偏差，因为工人较早提出的问题比后来提出的问题能得到更多后续工人的回答。在这里，我们研究如何利用这种不断增长的问题集进行高效的众包。通过将问题集建模为相互关联问题的网络，我们引入算法来通过在探索新问题和解决当前问题之间有效地分配工人来减少增长偏差。实验和模拟表明，这些算法可以有效地探索无界的问题集，而不会对群体答案失去信心。
