Diabetes mellitus is a serious metabolic disorder that can lead to a number of life-threatening complications. Studies have shown that intestinal microbiota is closely related to the development of diabetes, making it a potential target for the treatment of diabetes. In recent years, research on the active ingredients of traditional Chinese medicine (TCM), TCM compounds, and prepared Chinese medicines to regulate intestinal microbiota and improve the symptoms of diabetes mellitus is very extensive. We focus on the research progress of TCM active ingredients, herbal compounds, and prepared Chinese medicines in the treatment of diabetes mellitus in this paper. When diabetes occurs, changes in the abundance and function of the intestinal microbiota disrupt the intestinal environment by disrupting the intestinal barrier and fermentation. TCM and its components can increase the abundance of beneficial bacteria while decreasing the abundance of harmful bacteria, regulate the concentration of microbial metabolites, improve insulin sensitivity, regulate lipid metabolism and blood glucose, and reduce inflammation. TCM can be converted into active substances with pharmacological effects by intestinal microbiota, and these active substances can reverse intestinal microecological disorders and improve diabetes symptoms. This can be used as a reference for diabetes prevention and treatment.