The rumen is a vital organ containing vast amounts of microbes that play a key role in the digestion of nutrients and affect the production performance of ruminants. However, few studies have focused on the characterization of the ruminal microbiota composition and function in cows with long-term difference milk protein concentrations, and the relationship between milk protein concentration and ruminal microbiota remains elusive. In this study, we collected the data of milk protein concentrations of 1,025 Holstein cows for 10 mo on a commercial farm. Based on the milk protein concentrations, 30 cows were selected and divided into three groups (n = 10 per group): low milk protein group (LMP, milk protein concentration < 3.1%), medium milk protein group (MMP, 3.1% ≤ milk protein concentration < 3.4%), and high milk protein group (HMP, milk protein concentration ≥ 3.4%). The ruminal microbiome, metabolome, VFA concentrations and proportions, and amino acid profiles of the three groups were analyzed. The data showed that free amino acid (FAA) levels were lower in the rumen and higher in the plasma of HMP cows (P < 0.05). In addition, lower NH3 concentrations were observed in the rumen, plasma, and milk of the HMP cows (P < 0.05). Protease activity and isobutyric acid molar proportion in the rumen were lower in the HMP group (P < 0.05). Microbiome analysis showed that HMP cows had lower microbial diversity (represented as Shannon and Simpson indices) than LMP cows. At the genus level, lower relative abundances of Prevotella_1 and Ruminococcaceae_UCG_005 were observed in the HMP group (P < 0.05). At the operational taxonomic unit (OTU) level, a lower relative abundance of OTU3 (Prevotella ruminicola) was observed in the HMP group (P < 0.05). We found that the relative abundances of ruminal Prevotella_1 and OTU3 (Prevotella ruminicola) were negatively correlated with milk protein concentration (P < 0.05). These findings suggested that the cows with long-term high milk protein concentrations had lower microbial diversity and weaker protein degradation ability in the rumen. Furthermore, our observations identified a correlation between the milk protein concentration and ruminal microbiota.