Colorectal cancer (CRC) is the third most common cancer worldwide, and its incidence and mortality rates have been increasing annually in recent years. A variety of different small extracellular vesicles (sEVs) are important mediators of intercellular communication and have an important role in tumor metastasis and progression. The development and metastasis of CRC are closely linked to tumor-cell-derived sEVs, non-tumor-cell-derived sEVs, and intestinal-microbiota-derived sEVs. Numerous studies have shown that the tumor microenvironment (TME) is a key component in the regulation of CRC proliferation, development, and metastasis. These sEVs can create a TME conducive to CRC growth and metastasis by forming an immunosuppressive microenvironment, remodeling the extracellular matrix, and promoting tumor cell metabolism. Therefore, in this paper, we review the role of different types of sEVs in colorectal cancer development and metastasis. Furthermore, based on the properties of sEVs, we further discuss the use of sEVs as early biomarkers for colorectal cancer diagnosis and the potential for their use in the treatment of CRC.