Zhang Shunpu
Department of Statistics, University of Nebraska Lincoln, Lincoln, NE 68583-0963, USA.
BMC Bioinformatics. 2007 Jun 29;8:230. doi: 10.1186/1471-2105-8-230.
The Significance Analysis of Microarrays (SAM) is a popular method for detecting significantly expressed genes and controlling the false discovery rate (FDR). Recently, it has been reported in the literature that the FDR is not well controlled by SAM. Due to the vast application of SAM in microarray data analysis, it is of great importance to have an extensive evaluation of SAM and its associated R-package (sam2.20).
Our study has identified several discrepancies between SAM and sam2.20. One major difference is that SAM and sam2.20 use different methods for estimating FDR. Such discrepancies may cause confusion among the researchers who are using SAM or are developing the SAM-like methods. We have also shown that SAM provides no meaningful estimates of FDR and this problem has been corrected in sam2.20 by using a different formula for estimating FDR. However, we have found that, even with the improvement sam2.20 has made over SAM, sam2.20 may still produce erroneous and even conflicting results under certain situations. Using an example, we show that the problem of sam2.20 is caused by its use of asymmetric cutoffs which are due to the large variability of null scores at both ends of the order statistics. An obvious approach without the complication of the order statistics is the conventional symmetric cutoff method. For this reason, we have carried out extensive simulations to compare the performance of sam2.20 and the symmetric cutoff method. Finally, a simple modification is proposed to improve the FDR estimation of sam2.20 and the symmetric cutoff method.
Our study shows that the most serious drawback of SAM is its poor estimation of FDR. Although this drawback has been corrected in sam2.20, the control of FDR by sam2.20 is still not satisfactory. The comparison between sam2.20 and the symmetric cutoff method reveals that the relative performance of sam2.20 to the symmetric cutff method depends on the ratio of induced to repressed genes in a microarray data, and is also affected by the ratio of DE to EE genes and the distributions of induced and repressed genes. Numerical simulations show that the symmetric cutoff method has the biggest advantage over sam2.20 when there are equal number of induced and repressed genes (i.e., the ratio of induced to repressed genes is 1). As the ratio of induced to repressed genes moves away from 1, the advantage of the symmetric cutoff method to sam2.20 is gradually diminishing until eventually sam2.20 becomes significantly better than the symmetric cutoff method when the differentially expressed (DE) genes are either all induced or all repressed genes. Simulation results also show that our proposed simple modification provides improved control of FDR for both sam2.20 and the symmetric cutoff method.
微阵列显著性分析（SAM）是一种用于检测显著表达基因并控制错误发现率（FDR）的常用方法。最近，文献报道SAM对FDR的控制效果不佳。由于SAM在微阵列数据分析中的广泛应用，对SAM及其相关的R包（sam2.20）进行全面评估具有重要意义。
我们的研究发现了SAM和sam2.20之间的几个差异。一个主要区别是SAM和sam2.20使用不同的方法来估计FDR。这种差异可能会给使用SAM或开发类似SAM方法的研究人员带来困惑。我们还表明，SAM无法提供有意义的FDR估计值，而sam2.20通过使用不同的公式来估计FDR纠正了这个问题。然而，我们发现，即使sam2.20在SAM的基础上有所改进，但在某些情况下，sam2.20仍可能产生错误甚至相互矛盾的结果。通过一个例子，我们表明sam2.20的问题是由于其使用不对称截断值导致的，这是由于顺序统计量两端的零分变化很大。一种没有顺序统计量复杂性的明显方法是传统的对称截断方法。因此，我们进行了广泛的模拟，以比较sam2.20和对称截断方法的性能。最后，提出了一个简单的修改方法来改进sam2.20和对称截断方法的FDR估计。
我们的研究表明，SAM最严重的缺点是其对FDR的估计不佳。尽管这个缺点在sam2.20中得到了纠正，但sam2.20对FDR的控制仍然不令人满意。sam2.20与对称截断方法的比较表明，sam2.20相对于对称截断方法的相对性能取决于微阵列数据中诱导基因与抑制基因的比例，并且还受到差异表达（DE）基因与等效表达（EE）基因的比例以及诱导基因和抑制基因分布的影响。数值模拟表明，当诱导基因和抑制基因数量相等（即诱导基因与抑制基因的比例为1）时，对称截断方法相对于sam2.20具有最大优势。随着诱导基因与抑制基因的比例偏离1，对称截断方法相对于sam2.20的优势逐渐减小，直到最终当差异表达（DE）基因全部为诱导基因或全部为抑制基因时，sam2.20明显优于对称截断方法。模拟结果还表明，我们提出的简单修改方法为sam2.20和对称截断方法都提供了更好的FDR控制。
