Gegen Qinlian Decoction (GQD) is a Chinese herbal medicine that has been reported to significantly decrease blood glucose levels, which is suggested to be related to interactions with the gut microbiota. However, the protective effect of GQD on intestinal barrier function with regard to its influence on the gut microbiota has not been explored to date. In this study, we investigated the role of the gut microbiota in mediating the hypoglycemic mechanism of GQD in type 2 diabetes mellitus (T2DM) rats induced by a single intraperitoneal injection of streptozotocin after 4 weeks of high-fat diet feeding. The T2DM rats were randomly allocated to receive GQD, metformin (Met), or saline for 12 consecutive weeks, and changes in metabolic parameters, intestinal barrier function, and inflammation were investigated. Gut microbiota was analyzed using 16S rRNA gene sequencing from fecal samples, and statistical analyses were performed to correlate microbiota composition with phenotypes of the T2DM rats. GQD administration decreased the levels of blood glucose and inflammatory cytokines, and increased the levels of tight junction proteins. Besides, GQD had a protective effect on islet function, restoring intestinal permeability, and inhibiting inflammation, as evidenced by increases in the levels of serum C-peptide, occludin, and claudin-1 in the colon, and also improved the expression of serum inflammatory factors. In addition, GQD regulated the structure of the gut microbiota by increasing the proportions of short-chain fatty acids-producing and anti-inflammatory bacteria, and decreasing the proportions of conditioned pathogenic bacteria associated with the diabetic phenotype. Overall, these findings suggest that GQD could ameliorate hyperglycemia and protect islet function by regulating the structure of the gut microbiota, thereby restoring intestinal permeability and inhibiting inflammation in T2DM rats. Our study thus suggests that the hypoglycemic mechanism of GQD is mediated by its modulation of the gut microbiota.