The frequent occurrence of extreme climate events globally has elevated the requirements for environmental adaptability in livestock and poultry. Some goat populations have shown strong adaptability in specific extreme environments, and their genomes often leave genetic traces of adaptive evolution. This study integrated global goat single nucleotide polymorphism (SNP) chip data and raster data of 11 environmental variables. We retained 162 native goat populations and analyzed the environmental data of their regions. We detected 23 candidate genes related to environmental adaptation using selection signal analysis and genome-environment association analysis. After that, we screened out goat populations in extreme environments based on environmental data. Then, we used three selection signal analysis methods (F, XPEHH and θπ methods) to detect the genomes of these goat populations. In four different extreme environments (high elevation, hot, cold, and arid), 91, 43, 21, and 115 candidate genes were identified, respectively. Combined with studies related to environmental adaptation, we found that genes such as GULP1, GPC5, GPC6, and PDE4D may play important roles in the adaptation of goats to extreme environments. This study provides new insights into the adaptive mechanism of goats in extreme environments and provides an important theoretical basis for goat breed improvement and stress resistance breeding. At the same time, these findings also provide a reference for the study of the adaptability of other livestock in extreme environments.