Human respiratory syncytial viruses (RSVs) are classified into two major groups (A and B) based on antigenic differences in the G glycoprotein. To investigate circulating characteristics and phylodynamic history of RSV, we analyzed the genetic variability and evolutionary pattern of RSVs from 1977 to 2019 in this study. The results revealed that there was no recombination event of intergroup. Single nucleotide polymorphisms (SNPs) were observed through the genome with the highest occurrence rate in the G gene. Five and six sites in G protein of RSV-A and RSV-B, respectively, were further identified with a strong positive selection. The mean evolutionary rates for RSV-A and -B were estimated to be 1.48 × 10 and 1.92 × 10 nucleotide substitutions/site/year, respectively. The Bayesian skyline plot showed a constant population size of RSV-A and a sharp expansion of population size of RSV-B since 2005, and an obvious decrease 5 years later, then became stable again. The total population size of RSVs showed a similar tendency to that of RSV-B. Time-scaled phylogeny suggested a temporal specificity of the RSV-genotypes. Monitoring nucleotide changes and analyzing evolution pattern for RSVs could give valuable insights for vaccine and therapy strategies against RSV infection.