Traditional Chinese Medicine (TCM) has recently emerged as a beacon for the treatment of diabetes and its complications. Many TCMs that are commonly used, have the potentially demonstrated significant anti-diabetic effects. The mechanisms of these effects have been extensively discussed using modern techniques, such as genomics, mass spectrometry, and network pharmacology. Studies have demonstrated that TCM can influence glucose metabolism and pancreatic function via a diverse array of mechanisms including PI3K/AKT and AMPK pathways. TCM not only exhibits potential in the treatment of diabetes but also reduces the risk of diabetic complications. It is effective in the treatment of diabetic nephropathy (DN), diabetic retinopathy (DR), diabetic neuropathy (DPN), diabetic cardiomyopathy, and peripheral angiopathy. Research has demonstrated that prescriptions, Chinese herbal medicines, and their extracts play a role in a variety of molecular mechanisms such as antioxidation, apoptosis regulation, hypoxia improvement, autophagy, and promotion of glucose and lipid metabolism. The antioxidant properties of TCM have received considerable attention. Recent studies have demonstrated that they are capable of effectively eliminating free radicals from the body and reducing damage to cells caused by oxidative stress. Consequently, they are crucial in the treatment of diabetes and its associated complications. This review summarizes the ever-expanding scope of TCM applicability in the field of diabetes, providing crucial support and innovative ideas for modern healthcare. TCMs could help seek more effective pharmacological targets in basic study and as well serve as the complement to the strategy of diabetic prevention and treatment benefiting the patients. More and more large series of RCT and clinical investigations will eventually examine the efficacy of specific TCM formulas on the therapeutic effect of DM and its complication where currently treatments could not be satisfied.