Joint NTU-UBC Research Centre of Excellence in Active Living for the Elderly (LILY), Nanyang Technological University, Singapore, 639798, Singapore.
School of Computer Science and Engineering, Nanyang Technological University, Singapore, 639798, Singapore.
Sci Rep. 2017 Oct 2;7(1):12541. doi: 10.1038/s41598-017-12757-x.
Crowdsourcing systems are complex not only because of the huge number of potential strategies for assigning workers to tasks, but also due to the dynamic characteristics associated with workers. Maximizing social welfare in such situations is known to be NP-hard. To address these fundamental challenges, we propose the surprise-minimization-value-maximization (SMVM) approach. By analysing typical crowdsourcing system dynamics, we established a simple and novel worker desirability index (WDI) jointly considering the effect of each worker's reputation, workload and motivation to work on collective productivity. Through evaluating workers' WDI values, SMVM influences individual workers in real time about courses of action which can benefit the workers and lead to high collective productivity. Solutions can be produced in polynomial time and are proven to be asymptotically bounded by a theoretical optimal solution. High resolution simulations based on a real-world dataset demonstrate that SMVM significantly outperforms state-of-the-art approaches. A large-scale 3-year empirical study involving 1,144 participants in over 9,000 sessions shows that SMVM outperforms human task delegation decisions over 80% of the time under common workload conditions. The approach and results can help engineer highly scalable data-driven algorithmic management decision support systems for crowdsourcing.
众包系统不仅因为为任务分配工人的潜在策略数量巨大而复杂，还因为与工人相关的动态特征而复杂。在这种情况下，最大化社会效益被证明是 NP 难问题。为了解决这些基本挑战，我们提出了惊喜最小化-价值最大化（SMVM）方法。通过分析典型的众包系统动态，我们建立了一个简单而新颖的工人偏好指数（WDI），同时考虑每个工人的声誉、工作量和工作动机对集体生产力的影响。通过评估工人的 WDI 值，SMVM 实时影响个人工人的行动方案，这些方案可以使工人受益并提高集体生产力。解决方案可以在多项式时间内产生，并被证明渐近受理论最优解的限制。基于真实数据集的高分辨率模拟表明，SMVM 明显优于最先进的方法。一项涉及 1144 名参与者和超过 9000 个会话的大规模 3 年实证研究表明，在常见的工作量条件下，SMVM 在 80%以上的时间内优于人工任务分配决策。该方法和结果可以帮助工程师设计高度可扩展的数据驱动算法管理决策支持系统，以实现众包。
