Optimal strategies for sequential validation of significant features from high-dimensional genomic data.

High-dimensional genomic studies play a key role in identifying critical features that are significantly associated with a phenotypic outcome. The two most important examples are the detection of (1) differentially expressed genes from genome-wide gene expression studies and (2) single-nucleotide polymorphisms (SNPs) from genome-wide association studies. Such experiments are often associated with high noise levels, and the validity of statistical conclusions suffers from low sample size compared to large number of features. The corresponding multiple testing problem calls for the identification of optimal strategies for controlling the numbers of false discoveries and false nondiscoveries. In addition, a frequent validation problem is that features identified as important in one study are often less so in another study. Adjustment for multiple testing in both studies separately increases the risk of missing the crucial features even further. These problems can be addressed by sequential validation strategies, where only significant features identified in one study enter as candidates in the next study. The quality associated with different studies, for example, in terms of noise levels, may vary considerably. By performing simulation studies it is possible to demonstrate that the optimal order for this stepwise procedure is to sort experimental studies according to their quality in descending order. The impact of the method for multiple testing adjustment (Bonferroni-Holm, FDR) was also analyzed. Finally, the sequential validation strategy was applied to three large breast cancer studies with gene expression measurements, confirming the crucial impact of the order of the validation steps in a real-world application.