The gut microbiota is critical for host function. Among mammals, host phylogenetic relatedness and diet are strong drivers of gut microbiota structure, but one factor may be more influential than the other. Here, we used 16S rRNA gene sequencing to determine the relative contributions of host phylogeny and host diet in structuring the gut microbiotas of 11 herbivore species from 5 families living sympatrically in southwest Kenya. Herbivore species were classified as grazers, browsers, or mixed-feeders and dietary data (% C4 grasses in diet) were compiled from previously published sources. We found that herbivore gut microbiotas were highly species-specific, and that host taxonomy accounted for more variation in the gut microbiota (30%) than did host dietary guild (10%) or sample month (8%). Overall, similarity in the gut microbiota increased with host phylogenetic relatedness (r = 0.74) across the 11 species of herbivores, but among 7 closely related Bovid species, dietary %C4 grass values more strongly predicted gut microbiota structure (r = 0.64). Additionally, within bovids, host dietary guild explained more of the variation in the gut microbiota (17%) than did host species (12%). Lastly, while we found that the gut microbiotas of herbivores residing in southwest Kenya converge with those of distinct populations of conspecifics from central Kenya, fine-scale differences in the abundances of bacterial amplicon sequence variants (ASVs) between individuals from the two regions were also observed. Overall, our findings suggest that host phylogeny and taxonomy strongly structure the gut microbiota across broad host taxonomic scales, but these gut microbiotas can be further modified by host ecology (i.e., diet, geography), especially among closely related host species.