It is critical to identify the assembly processes and determinants of soil microbial communities to better predict soil microbial responses to environmental change in arid and semiarid areas. Here, soils from 16 grassland-only, 9 paired grassland and farmland, and 16 farmland-only sites were collected across the central Inner Mongolia Plateau, covering a steep environmental gradient. Through analyzing the paired samples, we discovered that land uses had strong effects on soil microbial communities but weak effects on their assembly processes. For all samples, although no environmental variables were significantly correlated with the net relatedness index (NRI), both the nearest taxon index (NTI) and the β-nearest taxon index (βNTI) were most related to mean annual precipitation (MAP). With the increase of MAP, soil microbial taxa at the tips of the phylogenetic tree were more clustered, and the contribution of determinism increased. Determinism (48.6%), especially variable selection (46.3%), and stochasticity (51.4%) were almost equal in farmland, while stochasticity (75.0%) was dominant in grassland. Additionally, Mantel tests and redundancy analyses (RDA) revealed that the main determinants of soil microbial community structure were MAP in grassland but mean annual temperature (MAT) in farmland. MAP and MAT were also good predictors of the community composition (the top 200 dominant operational taxonomic units) in grassland and farmland, respectively. Collectively, in arid and semiarid areas, soil microbial communities were more sensitive to environmental change in farmland than in grassland, and unlike the major impact of MAP on grassland microbial communities, MAT was the primary driver of farmland microbial communities. As one of the most diverse organisms, soil microbes play indispensable roles in many ecological processes in arid and semiarid areas with limited macrofaunal and plant diversity, yet the mechanisms underpinning soil microbial community are not fully understood. In this study, soil microbial communities were investigated along a 500-km transect covering a steep environmental gradient across farmland and grassland in the areas. The results showed that precipitation was the main factor mediating the assembly processes. Determinism was more influential in farmland, and variable selection of farmland was twice that of grassland. Temperature mainly drove farmland microbial communities, while precipitation mainly affected grassland microbial communities. These findings provide new information about the assembly processes and determinants of soil microbial communities in arid and semiarid areas, consequently improving the predictability of the community dynamics, which have implications for sustaining soil microbial diversity and ecosystem functioning, particularly under global climate change conditions.