Type 2 diabetes mellitus (T2DM) is one of the most risk factors threatening human health. Although genetic and environmental factors contribute to the development of T2DM, gut microbiota has also been found to be involved. Gut microbiota-derived metabolites are a key factor in host-microbe crosstalk, and have been revealed to play a central role in the physiology and physiopathology of T2DM. In this review, we provide a timely and comprehensive summary of the microbial metabolites that are protective or causative for T2DM, including some amino acids-derived metabolites, short-chain fatty acids, trimethylamine N-oxide, and bile acids. The mechanisms by which metabolites affect T2DM have been elaborated. Knowing more about these processes will increase our understanding of the causal relationship between gut microbiota and T2DM. Moreover, some frontier therapies that target gut microbes and their metabolites to improve T2DM, including dietary intervention, fecal microbiota transplantation, probiotics, prebiotics or synbiotics intervention, and drugging microbial metabolism, have been critically discussed. This review may provide novel insights for the development of targeted and personalized treatments for T2DM based on gut microbial metabolites. More high-quality clinical trials are needed to accelerate the clinical translation of gut-targeted therapies for T2DM.