IEEE Trans Pattern Anal Mach Intell. 2021 Jan;43(1):334-346. doi: 10.1109/TPAMI.2019.2922396. Epub 2020 Dec 4.
In this paper, we study weakly supervised learning where a large amount of data supervision is not accessible. This includes i) incomplete supervision, where only a small subset of labels is given, such as semi-supervised learning and domain adaptation; ii) inexact supervision, where only coarse-grained labels are given, such as multi-instance learning and iii) inaccurate supervision, where the given labels are not always ground-truth, such as label noise learning. Unlike supervised learning which typically achieves performance improvement with more labeled examples, weakly supervised learning may sometimes even degenerate performance with more weakly supervised data. Such deficiency seriously hinders the deployment of weakly supervised learning to real tasks. It is thus highly desired to study safe weakly supervised learning, which never seriously hurts performance. To this end, we present a generic ensemble learning scheme to derive a safe prediction by integrating multiple weakly supervised learners. We optimize the worst-case performance gain and lead to a maximin optimization. This brings multiple advantages to safe weakly supervised learning. First, for many commonly used convex loss functions in classification and regression, it is guaranteed to derive a safe prediction under a mild condition. Second, prior knowledge related to the weight of the base weakly supervised learners can be flexibly embedded. Third, it can be globally and efficiently addressed by simple convex quadratic or linear program. Finally, it is in an intuitive geometric interpretation with the least square loss. Extensive experiments on various weakly supervised learning tasks, including semi-supervised learning, domain adaptation, multi-instance learning and label noise learning demonstrate our effectiveness.
在本文中，我们研究了弱监督学习，其中大量数据的监督是不可用的。这包括 i）不完全监督，其中只给出了一小部分标签，例如半监督学习和领域自适应；ii）不精确监督，其中只给出了粗粒度的标签，例如多实例学习；iii）不准确监督，其中给定的标签并不总是真实的，例如标签噪声学习。与通常通过更多带标签的示例来提高性能的监督学习不同，弱监督学习有时甚至会随着更多弱监督数据的出现而降低性能。这种不足严重阻碍了弱监督学习在实际任务中的部署。因此，研究安全的弱监督学习非常重要，即永远不会严重损害性能。为此，我们提出了一种通用的集成学习方案，通过集成多个弱监督学习器来得出安全的预测。我们优化了最坏情况下的性能增益，并导致了最大化最小化优化。这为安全的弱监督学习带来了多个优势。首先，对于分类和回归中常用的许多凸损失函数，在一个温和的条件下，它可以保证得出安全的预测。其次，可以灵活地嵌入与基弱监督学习器权重相关的先验知识。第三，它可以通过简单的凸二次或线性规划全局且高效地解决。最后，它具有直观的几何解释，与最小二乘损失一致。在各种弱监督学习任务（包括半监督学习、领域自适应、多实例学习和标签噪声学习）上的广泛实验表明了我们方法的有效性。
