IEEE Trans Pattern Anal Mach Intell. 2018 Mar;40(3):582-594. doi: 10.1109/TPAMI.2017.2682082. Epub 2017 Mar 15.
Active learning is an effective way of engaging users to interactively train models for visual recognition more efficiently. The vast majority of previous works focused on active learning with a single human oracle. The problem of active learning with multiple oracles in a collaborative setting has not been well explored. We present a collaborative computational model for active learning with multiple human oracles, the input from whom may possess different levels of noises. It leads to not only an ensemble kernel machine that is robust to label noises, but also a principled label quality measure to online detect irresponsible labelers. Instead of running independent active learning processes for each individual human oracle, our model captures the inherent correlations among the labelers through shared data among them. Our experiments with both simulated and real crowd-sourced noisy labels demonstrate the efficacy of our model.
主动学习是一种有效的方法，可以让用户参与到视觉识别模型的交互训练中，从而更有效地提高效率。以前的绝大多数工作都集中在使用单个人类标注者进行主动学习上。在协作环境中使用多个标注者的主动学习问题还没有得到很好的探索。我们提出了一个用于多个人类标注者的主动学习的协作计算模型，他们的输入可能具有不同程度的噪声。这不仅导致了对标签噪声具有鲁棒性的集成核机器，而且还提出了一种用于在线检测不负责任的标注者的原则性标签质量度量。我们的模型不是为每个单独的人类标注者运行独立的主动学习过程，而是通过它们之间的共享数据来捕捉标注者之间的内在相关性。我们使用模拟和真实的众包噪声标签进行的实验证明了我们模型的有效性。
