The exploitation of ion-adsorption rare earth elements (REEs) deposits results in serious ecological and environmental problems, which has attracted much attention. However, the influences of exploitation on the prokaryotic communities and their complex interactions remain poorly understood. In the present study, bacterial and archaeal communities, as well as ammonia-oxidizing bacteria (AOB) and ammonia-oxidizing archaea (AOA), in and around REEs mining area were investigated through high throughput sequencing and quantitative polymerase chain reaction (qPCR). Our results indicated that mining soil was characterized by poor soil structure, nutrient deficiency, and high concentrations of residual REEs. Oligotrophic bacteria (e.g., Chloroflexi and Acidobacteriota) were dominant in unexploited soil and mining soil, while copiotrophic bacteria (Proteobacteria and Actinobacteriota) were more abundant in surrounding soil. Nutrient was the key factor affecting microbial variation and abundance in mining soil. The bacterial community was more sensitive to REEs, while the archaeal communities were relatively stable. As the key members for ammonia oxidation, AOA outnumbered AOB in all the soil types, and the former was significantly influenced by pH, nutrients, and TREEs in mining soil. The microbial co-occurrence network analysis demonstrated that exploitation significantly influenced topological properties, decreased the complexity, and resulted in a much unstable network, leading to a more fragile microbial ecosystem in mining areas. Notably, the abundance of keystone taxa decreased after exploitation, and oligotrophic groups (Chloroflexi) replaced copiotrophic groups (Proteobacteria and Actinobacteriota) as the key to rebuilt a co-occurrence network, suggesting potentially important roles in maintaining network stability. The current results are of great significance to the ecological risk assessment of REEs exploitation.