Fungal residue is a unique abundant organic material undervalued in agricultural production. The application of chemical fertilizer combined with fungal residue can not only improve soil quality but also regulate the microbial community. However, it is unclear whether the response of soil bacteria and fungi to the combined application of fungal residue and chemical fertilizer is consistent. Therefore, a long-term positioning experiment in a rice field was conducted with a total of nine treatments. Chemical fertilizer (C) and fungal residue (F) were applied at 0, 50%, and 100% to evaluate 1 the change in soil fertility properties and microbial community structure and 2 the main driving factors of soil microbial diversity and species composition. The results showed that soil total nitrogen (TN) was highest after treatment CF (55.56% higher than in the control), and the carbon to nitrogen ratio (C/N), total phosphorus (TP), dissolved organic carbon (DOC), and available phosphorus (AP) contents were highest after treatment with CF(26.18%, 26.46%, 17.13%, and 279.54% higher than in the control, respectively). The amounts of soil organic carbon (SOC), available nitrogen (AN), available potassium (AK), and pH were highest after treatment with CF (85.57%, 41.61%, 29.33%, and 4.62% higher than in the control, respectively). Following the application of fungal residue with chemical fertilizer, there were significant changes in the -diversity of bacteria and fungi in each treatment. Compared with that of the control (CF), different long-term applications of fungal residue with chemical fertilizer did not significantly change soil bacterial -diversity but resulted in significant differences in fungal -diversity, and the relative abundance of soil fungal Ascomycota and Sordariomycetes significantly decreased after the application of CF. The random forest prediction model indicated that AP and C/N were the main driving factors of bacterial and fungal -diversity, respectively, and AN, pH, SOC, and DOC were the main driving factors of bacterial -diversity, whereas AP and DOC were the main driving factors of fungal -diversity. Correlation analysis suggested that the relative abundance of soil fungal Ascomycota and Sordariomycetes had a significantly negative correlation with SOC, TN, TP, AN, AP, AK, and C/N. PERMANOVA showed that variation in soil fertility properties, dominant species of soil bacteria at the phylum and class level, and dominant species of soil fungi at the phylum and class level were all best explained by fungal residue (46.35%, 18.47%, and 41.57%, respectively), and variation in bacterial diversity was best explained by fungal residue (23.84%) and to a lesser extent by the interaction between fungal residue and chemical fertilizer (9.90%). In contrast, the variation in fungal diversity was best explained by the interaction between fungal residue and chemical fertilizer (35.00%) and to a lesser extent by fungal residue (10.42%). In conclusion, the application of fungal residue has more advantages than chemical fertilizer in influencing soil fertility properties and microbial community structure changes.