Tumor hypoxia drives metabolic shifts, cancer progression, and therapeutic resistance. Challenges in quantifying hypoxia have hindered the exploitation of this potential "Achilles' heel." While gene expression signatures have shown promise as surrogate measures of hypoxia, signature usage is heterogeneous and debated. Here, we present a systematic pan-cancer evaluation of 70 hypoxia signatures and 14 summary scores in 104 cell lines and 5,407 tumor samples using 472 million length-matched random gene signatures. Signature and score choice strongly influenced the prediction of hypoxia in vitro and in vivo. In cell lines, the Tardon signature was highly accurate in both bulk and single-cell data (94% accuracy, interquartile mean). In tumors, the Buffa and Ragnum signatures demonstrated superior performance, with Buffa/mean and Ragnum/interquartile mean emerging as the most promising for prospective clinical trials. This work delivers recommendations for experimental hypoxia detection and patient stratification for hypoxia-targeting therapies, alongside a generalizable framework for signature evaluation.