Wu Jin Chu, Martin Alvin F, Greenberg Craig S, Kacker Raghu N
National Institute of Standards and Technology, Gaithersburg, MD 20899 USA.
IEEE/ACM Trans Audio Speech Lang Process. 2017 Jan;25(1):5-18. doi: 10.1109/TASLP.2016.2614725. Epub 2016 Sep 30.
The data dependency due to multiple use of the same subjects has impact on the standard error (SE) of the detection cost function (DCF) in speaker recognition evaluation. The DCF is defined as a weighted sum of the probabilities of type I and type II errors at a given threshold. A two-layer data structure is constructed: target scores are grouped into target sets based on the dependency, and likewise for non-target scores. On account of the needed equal probabilities for scores being selected when resampling, target sets must contain the same number of target scores, and so must non-target sets. In addition to the bootstrap method with i.i.d. assumption, the nonparametric two-sample one-layer and two-layer bootstrap methods are carried out based on whether the resampling takes place only on sets, or subsequently on scores within the sets. Due to the stochastic nature of the bootstrap, the distributions of the SEs of the DCF estimated using the three different bootstrap methods are created and compared. After performing hypothesis testing, it is found that data dependency increases not only the SE but also the variation of the SE, and the two-layer bootstrap is more conservative than the one-layer bootstrap. The rationale regarding the different impacts of the three bootstrap methods on the estimated SEs is investigated.
在说话人识别评估中，由于同一主体的多次使用所导致的数据依赖性会对检测成本函数（DCF）的标准误差（SE）产生影响。DCF被定义为在给定阈值下第一类错误和第二类错误概率的加权和。构建了一种两层数据结构：基于依赖性将目标分数分组为目标集，非目标分数也同样处理。由于重采样时选择分数所需的概率相等，目标集必须包含相同数量的目标分数，非目标集也必须如此。除了具有独立同分布假设的自助法之外，基于重采样是仅在集合上进行还是随后在集合内的分数上进行，还实施了非参数双样本单层和两层自助法。由于自助法的随机性，创建并比较了使用三种不同自助法估计的DCF的SE分布。进行假设检验后发现，数据依赖性不仅会增加SE，还会增加SE的变化，并且两层自助法比单层自助法更保守。研究了三种自助法对估计SE的不同影响的原理。
