The microbial composition of the rhizosphere soil could be an important determinant of crop yield, pathogen resistance, and other beneficial attributes in plants. However, little is known about the impact of cropping sequences on microbial community dynamics, especially in economically important species like soybean. Using 2-year crop sequences of corn-soybean, canola-soybean, and soybean-soybean, we investigated how crops from the previous growing season influenced the structure of the microbiome in both the bulk soil and soybean rhizosphere. A combination of marker-based Illumina sequencing and bioinformatics analyses was used to show that bacterial species richness and evenness in the soybean rhizosphere soil were similar following canola and soybean compared to a previous corn sequence. However, fungal species richness and evenness remained unaffected by crop sequence. In addition, bacterial and fungal species diversity in both the bulk and soybean rhizosphere soil were not influenced by crop sequence. Lastly, the corn-soybean sequence significantly differed in the relative abundance of certain bacterial and fungal classes in both the soybean rhizosphere and bulk soil. While canola-soybean and a continuous soybean sequence did not, suggesting that a preceding corn sequence may reduce the occurrence of overall bacterial and fungal community members. For the present study, crop sequence impacts bacterial diversity and richness in both the bulk soil and soybean rhizosphere soil whereas fungal diversity and richness are resilient to crop sequence practices. Together, these findings could help drive decision making for annual crop and soil management practices.