The outbreak of COVID-19 has become a global crisis, and brought severe disruptions to societies and economies. Until now, effective therapeutics against COVID-19 are in high demand. Along with our improved understanding of the structure, function, and pathogenic process of SARS-CoV-2, many small molecules with potential anti-COVID-19 effects have been developed. So far, several antiviral strategies were explored. Besides directly inhibition of viral proteins such as RdRp and M, interference of host enzymes including ACE2 and proteases, and blocking relevant immunoregulatory pathways represented by JAK/STAT, BTK, NF-κB, and NLRP3 pathways, are regarded feasible in drug development. The development of small molecules to treat COVID-19 has been achieved by several strategies, including computer-aided lead compound design and screening, natural product discovery, drug repurposing, and combination therapy. Several small molecules representative by remdesivir and paxlovid have been proved or authorized emergency use in many countries. And many candidates have entered clinical-trial stage. Nevertheless, due to the epidemiological features and variability issues of SARS-CoV-2, it is necessary to continue exploring novel strategies against COVID-19. This review discusses the current findings in the development of small molecules for COVID-19 treatment. Moreover, their detailed mechanism of action, chemical structures, and preclinical and clinical efficacies are discussed.