Wei Hongxin, Xie Renchunzi, Feng Lei, Han Bo, An Bo
IEEE Trans Neural Netw Learn Syst. 2023 Dec;34(12):10552-10562. doi: 10.1109/TNNLS.2022.3168696. Epub 2023 Nov 30.
Crowdsourcing is a popular solution for large-scale data annotations. So far, various end-to-end deep learning methods have been proposed to improve the practical performance of learning from crowds. Despite their practical effectiveness, most of them have two major limitations-they do not hold learning consistency and suffer from computational inefficiency. In this article, we propose a novel method named UnionNet, which is not only theoretically consistent but also experimentally effective and efficient. Specifically, unlike existing methods that either fit a given label from each annotator independently or fuse all the labels into a reliable one, we concatenate the one-hot encoded vectors of crowdsourced labels provided by all the annotators, which takes all the labeling information as a union and coordinates multiple annotators. In this way, we can directly train an end-to-end deep neural network by maximizing the likelihood of this union with only a parametric transition matrix. We theoretically prove the learning consistency and experimentally show the effectiveness and efficiency of our proposed method.
众包是大规模数据标注的一种流行解决方案。到目前为止，已经提出了各种端到端深度学习方法来提高从众包中学习的实际性能。尽管它们具有实际有效性，但大多数方法有两个主要局限性——它们不具备学习一致性且存在计算效率低下的问题。在本文中，我们提出了一种名为UnionNet的新方法，它不仅在理论上具有一致性，而且在实验上有效且高效。具体而言，与现有方法不同，现有方法要么独立地拟合每个标注者给出的给定标签，要么将所有标签融合成一个可靠的标签，我们将所有标注者提供的众包标签的独热编码向量连接起来，将所有标注信息视为一个并集，并协调多个标注者。通过这种方式，我们可以通过仅使用一个参数转移矩阵最大化这个并集的似然性来直接训练一个端到端深度神经网络。我们在理论上证明了学习一致性，并在实验上展示了我们所提出方法的有效性和效率。
