Wang Yongqiao, Li Lishuai, Dang Chuangyin
IEEE Trans Pattern Anal Mach Intell. 2019 Aug;41(8):1813-1827. doi: 10.1109/TPAMI.2019.2895794. Epub 2019 Jan 28.
In many real-world classification problems, accurate prediction of membership probabilities is critical for further decision making. The probability calibration problem studies how to map scores obtained from one classification algorithm to membership probabilities. The requirement of non-decreasingness for this mapping involves an infinite number of inequality constraints, which makes its estimation computationally intractable. For the sake of this difficulty, existing methods failed to achieve four desiderata of probability calibration: universal flexibility, non-decreasingness, continuousness and computational tractability. This paper proposes a method with shape-restricted polynomial regression, which satisfies all four desiderata. In the method, the calibrating function is approximated with monotone polynomials, and the continuously-constrained requirement of monotonicity is equivalent to some semidefinite constraints. Thus, the calibration problem can be solved with tractable semidefinite programs. This estimator is both strongly and weakly universally consistent under a trivial condition. Experimental results on both artificial and real data sets clearly show that the method can greatly improve calibrating performance in terms of reliability-curve related measures.
在许多现实世界的分类问题中，准确预测隶属概率对于进一步的决策至关重要。概率校准问题研究如何将从一种分类算法获得的分数映射到隶属概率。这种映射的非递减要求涉及无限数量的不等式约束，这使得其估计在计算上难以处理。鉴于此困难，现有方法未能实现概率校准的四个期望特性：通用灵活性、非递减性、连续性和计算易处理性。本文提出了一种具有形状受限多项式回归的方法，该方法满足所有四个期望特性。在该方法中，校准函数用单调多项式近似，并且单调性的连续约束要求等同于一些半定约束。因此，校准问题可以用易于处理的半定规划来解决。在一个平凡条件下，该估计器既是强通用一致的，也是弱通用一致的。在人工数据集和真实数据集上的实验结果清楚地表明，该方法在与可靠性曲线相关的度量方面可以大大提高校准性能。
