Studying shifts in microbial communities under different land use can help in determining the impact of land use on microbial diversity. In this study, we analyzed four different land-use types to determine their bacterial and archaeal diversity and abundance. Three natural ecosystems, that is, wetland (WL), grassland (GL), and forest (FR) soils, and one agricultural soil, that is, tea plantation (TP) soil, were investigated to determine how land use shapes bacterial and archaeal diversity. For this purpose, molecular analyses, such as quantitative polymerase chain reaction (Q-PCR), 16S rRNA gene sequencing, and terminal restriction fragment length polymorphism (T-RFLP), were used. Soil physicochemical properties were determined, and statistical analyses were performed to identify the key factors affecting microbial diversity in these soils. Phylogenetic affiliations determined using the Ribosomal Database Project (RDP) database and T-RFLP revealed that the soils had differing bacterial diversity. WL soil was rich in only Proteobacteria, whereas GR soil was rich in Proteobacteria, followed by Actinobacteria. FR soil had higher abundance of Chloroflexi species than these soils. TP soil was rich in Actinobacteria, followed by Chloroflexi, Acidobacteria, Proteobacteria, and Firmicutes. The archaeal diversity of GL and FR soils was similar in that most of their sequences were closely related to Nitrososphaerales (Thaumarchaeota phylum). In contrast, WL soil, followed by TP soil, had greater archaeal diversity than other soils. Eight different archaeal classes were found in WL soil, and Pacearchaeota class was the richest one. The abundance of bacterial and archaeal 16S rRNA gene copies in WL and GL soils was significantly higher than that in FR and TP soils. Redundancy analysis showed that bacterial diversity was influenced by abiotic factors, e.g., total organic carbon and pH, whereas total nitrogen, pH, and cation exchange capacity (CEC) significantly affected archaeal community composition. Pearson correlation analysis showed that bacterial and archaeal 16S rRNA gene abundance had the highest correlation with clay content (r > 0.905, P < 0.01), followed by total-P, CEC, pH, and silt (%). These results will lead to more comprehensive understanding of how land use affects microbial distribution.