Genazzani A D, Rodbard D
Laboratory of Theoretical and Physical Biology, National Institutes of Child Health and Human Development, National Institutes of Health, Bethesda, Maryland.
Acta Endocrinol (Copenh). 1991 Mar;124(3):295-306. doi: 10.1530/acta.0.1240295.
We utilize the "Receiver Operating Characteristic" to describe the relationship between sensitivity and specificity as the threshold for peak detection is varied systematically, to provide objective comparison of the performance of methods for detection of episodic hormonal secretion. A computer program was used to generate synthetic data with peaks with variable durations, with constant or variable height, shape and/or interpulse interval. This approach was used to compare the CLUSTER and DETECT programs. For both programs, the observed false positive rates estimated using signal-free data were in good agreement with the nominal rates, but in the presence of signal the observed false positive rates were systematically lower. Sensitivity increases with increasing signal/noise ratio, as expected. Program DETECT, using its standard options, provided excellent sensitivity (90-100%) with very low false positive rate under all conditions tested. Its performance could be further improved by the use of a more stringent definition of a peak requiring the presence of "UP" followed by a "DOWN". The CLUSTER program was found to have very poor sensitivity when using the "local variance" option. Use of the true fixed standard deviation or percent coefficient of variation resulted in a modest improvement. Optimal performance of program CLUSTER was obtained by the use of the best of 3 variance models, testing 12 different cluster sizes (from 1x1 to 4x4) and selecting the best among these: under these conditions it can achieve high sensitivity (90-100%) for very low observed false positive rate, such that its performance was comparable to that of DETECT. The methods developed and illustrated here should permit the definitive characterization and validation of the performance of any one method, the objective comparison of the relative performance of two or more methods for analysis of pulsatile hormone levels for episodic hormone secretion, and lead to the improvement of algorithms for peak detection.
我们利用“受试者工作特征曲线”来描述灵敏度和特异性之间的关系，因为峰值检测的阈值是系统变化的，以此来客观比较检测间歇性激素分泌的方法的性能。使用一个计算机程序来生成具有可变持续时间、恒定或可变高度、形状和/或脉冲间期的峰值的合成数据。这种方法用于比较CLUSTER和DETECT程序。对于这两个程序，使用无信号数据估计的观察到的假阳性率与标称率吻合良好，但在有信号的情况下，观察到的假阳性率系统性地更低。正如预期的那样，灵敏度随着信噪比的增加而提高。使用其标准选项的DETECT程序在所有测试条件下都具有出色的灵敏度（90 - 100%），假阳性率非常低。通过使用更严格的峰值定义，即要求先出现“上升”然后是“下降”，其性能可以进一步提高。发现CLUSTER程序在使用“局部方差”选项时灵敏度非常低。使用真实的固定标准差或变异系数百分比会有一定程度的改善。通过使用3种方差模型中的最佳模型、测试12种不同的聚类大小（从1x1到4x4）并从中选择最佳模型，CLUSTER程序获得了最佳性能：在这些条件下，它可以实现高灵敏度（90 - 100%），观察到的假阳性率非常低，以至于其性能与DETECT相当。这里开发和说明的方法应该能够对任何一种方法的性能进行明确的表征和验证，对两种或更多种分析间歇性激素分泌的脉动激素水平的方法的相对性能进行客观比较，并导致峰值检测算法的改进。
