Mounting evidence suggested that the gut microbiota has a significant role in the metabolism and disease status of the host. In particular, Type 2 Diabetes (T2D), which has a complex etiology that includes obesity and chronic low-grade inflammation, is modulated by the gut microbiota and microbial metabolites. Current literature supports that unbalanced gut microbial composition (dysbiosis) is a risk factor for T2D. In this review, we critically summarize the recent findings regarding the role of gut microbiota in T2D. Beyond these associative studies, we focus on the causal relationship between microbiota and T2D established using fecal microbiota transplantation (FMT) or probiotic supplementation, and the potential underlying mechanisms such as byproducts of microbial metabolism. These microbial metabolites are small molecules that establish communication between microbiota and host cells. We critically summarize the associations between T2D and microbial metabolites such as short-chain fatty acids (SCFAs) and trimethylamine N-Oxide (TMAO). Additionally, we comment on how host genetic architecture and the epigenome influence the microbial composition and thus how the gut microbiota may explain part of the missing heritability of T2D found by GWAS analysis. We also discuss future directions in this field and how approaches such as FMT, prebiotics, and probiotics supplementation are being considered as potential therapeutics for T2D.