Biometric Research Branch, National Cancer Institute, 9000 Rockville Pike, MSC 7434, Bethesda, MD 20892-7434, USA.
Biostatistics. 2011 Jul;12(3):399-412. doi: 10.1093/biostatistics/kxq069. Epub 2010 Nov 17.
For medical classification problems, it is often desirable to have a probability associated with each class. Probabilistic classifiers have received relatively little attention for small n large p classification problems despite of their importance in medical decision making. In this paper, we introduce 2 criteria for assessment of probabilistic classifiers: well-calibratedness and refinement and develop corresponding evaluation measures. We evaluated several published high-dimensional probabilistic classifiers and developed 2 extensions of the Bayesian compound covariate classifier. Based on simulation studies and analysis of gene expression microarray data, we found that proper probabilistic classification is more difficult than deterministic classification. It is important to ensure that a probabilistic classifier is well calibrated or at least not "anticonservative" using the methods developed here. We provide this evaluation for several probabilistic classifiers and also evaluate their refinement as a function of sample size under weak and strong signal conditions. We also present a cross-validation method for evaluating the calibration and refinement of any probabilistic classifier on any data set.
对于医学分类问题，通常希望为每个类别关联一个概率。尽管在医学决策中具有重要意义，但概率分类器在小 n 大 p 分类问题中并没有受到太多关注。在本文中，我们引入了两个用于评估概率分类器的标准：校准良好和细化，并开发了相应的评估指标。我们评估了几种已发表的高维概率分类器，并开发了贝叶斯复合协变量分类器的两种扩展。基于模拟研究和基因表达微阵列数据分析，我们发现正确的概率分类比确定性分类更困难。使用这里开发的方法确保概率分类器校准良好或至少不“保守”非常重要。我们为几种概率分类器提供了这种评估，并还评估了它们在弱信号和强信号条件下随样本量的细化情况。我们还提出了一种交叉验证方法，用于评估任何概率分类器在任何数据集上的校准和细化情况。
