In the present study, complete genomic sequences retrieved from 57 rhizobial strains that covered four genera including 11 species were analyzed comprehensively. The four types of replicons: chromosomes, chromids, nonsymbiotic plasmids, and symbiotic plasmids were investigated and compared among these strains. Results showed that co-evolution occurred among these four replicons based on the similarities in average nucleotide identity. High correlation coefficient r values were observed between chromosomes and chromids, as well as between chromosomes and nonsymbiotic plasmids. Chromosomes and symbiotic plasmids showed different phylogenetic topology based on their core genes. Population structure analyses were performed to extrapolate the evolutionary histories of the test strains based on their chromosomal and symbiotic plasmid background. This resulted in seven ancestral types for chromosomal genes and three ancestral types for symbiotic plasmid genes. Rhizobial strains containing chromosome genes with ancestral type E tend to contain symbiotic plasmid genes with ancestral type II, while rhizobial strains containing chromosome genes with ancestral type G tend to contain symbiotic plasmid genes with ancestral type III. Seventeen strains associated with different host plant species which harbored the symbiotic genes with ancestral type I, exhibited high genetic diversity. In addition, Fu's test of the symbiotic plasmid genes with ancestral type III had undergone an expansion event, implying the influence of negative selection on these symbiotic plasmid genes.