As a vital component of airborne bioaerosols, bacteria and fungi seriously endanger human health as pathogens and allergens. However, comprehensive effects of environmental variables on airborne microbial community structures remain poorly understood across the PM sizes and seasons. We collected atmospheric PM, PM, and PM samples in Hefei, a typical rapidly-developing city in East China, across three seasons, and performed a comprehensive analysis of airborne microbial community structures using qPCR and high-throughput sequencing. Overall the bacterial and fungal abundances in PM were one to two orders of magnitude higher than those in PM and PM across seasons, but their α-diversity tended to increase from PM to PM. The bacterial gene abundances showed a strong positive correlation (P < 0.05) with atmospheric SO and NO concentrations and air quality index. The bacterial gene abundances were significantly higher (P = 0.001) than fungi, and the bacterial diversity showed stronger seasonality. The PM sizes influenced distribution patterns for airborne microbial communities within the same season. Source-tracking analysis indicated that soils, plants, human and animal feces represented important sources of airborne bacteria with a total relative abundance of more than 60% in summer, but total abundance from the unidentified sources surpassed in fall and winter. Total 10 potential bacterial and 12 potential fungal pathogens were identified at the species level with the highest relative abundances in summer, and their abundances increased with the PM sizes. Together, our results indicated that a complex set of environmental factors, including water-soluble ions in PM, changes in air pollutant levels and meteorological conditions, and shifts in the relative importance of available microbial sources, acted to control the seasonal compositions of microbial communities in the urban atmosphere.