Generalized mixture models for molecular phylogenetic estimation.

The rapidly growing availability of multigene sequence data during the past decade has enabled phylogeny estimation at phylogenomic scales. However, dealing with evolutionary process heterogeneity across the genome becomes increasingly challenging. Here we develop a mixture model approach that uses reversible jump Markov chain Monte Carlo (MCMC) estimation to permit as many distinct models as the data require. Each additional model considered may be a fully parametrized general time-reversible model or any of its special cases. Furthermore, we expand the usual proposal mechanisms for topology changes to permit hard polytomies (i.e., zero-length internal branches). This new approach is implemented in the Crux software toolkit. We demonstrate the feasibility of using reversible jump MCMC on mixture models by reexamining a well-known 44-taxon mammalian data set comprising 22 concatenated genes. We are able to reproduce the results of the original analysis (with respect to bipartition support) when we make identical assumptions, but when we allow for polytomies and/or use data-driven mixture model estimation, we infer much lower bipartition support values for several key bipartitions.