Ruminal microflora is closely correlated with the ruminant's diet. However, information regarding the effect of high concentrate diets on rumen microflora in yaks is lacking. In the current study, 24 healthy male yaks were randomly assigned to two groups, each fed with different diets: less concentrate (LC; concentrate: coarse = 40: 60) and high concentrate (HC; concentrate: coarse = 80: 20) diets. Subsequently, a 21-day feeding trial was performed with the yaks, and rumen fluid samples were collected and compared using 16 s rRNA sequencing. The results showed that NH-N, total VFA, acetate, butyrate, isobutyrate, and isovalerate were significantly higher in the HC group than that in the LC group ( < 0.05), while microbial diversity and richness were significantly lower in the HC group ( < 0.05). Principal coordinate analysis indicated that rumen microflora was significantly different in LC and HC groups ( < 0.05). In the rumen, phyla Firmicutes and Bacteroidota were the most abundant bacteria, with Firmicutes being more abundant, and Bacteroidota being less abundant in the HC group than those found in the LC group. and are the highest abundant ones at the genus level. The relative abundance of , , and were significantly higher in the HC group than that in the LC group ( < 0.05), while the relative abundance of was significantly lower in the HC group than in the LC group ( < 0.05). Compared to the LC group, the relative abundance of , , and was significantly higher in the HC group. The relative abundances of , , , , , , and were correlated with ruminal fermentation parameters ( < 0.05). Furthermore, PICRUSt 2 estimation indicated that microbial genes associated with valine, leucine, and isoleucine biosynthesis were overexpressed in the rumen microflora of yaks in the HC group ( < 0.05). Conclusively, our results suggest that high concentrate diets affect the microflora composition and fermentation function in yak rumen. The present findings would provide new insights into the health of yaks under high concentrate feeding conditions and serve as a potent reference for the short-term fattening processes of yaks.