As the COVID-19 pandemic continues to unfold, the morbidity and mortality are increasing daily. Effective treatment for SARS-CoV-2 is urgently needed. We recently discovered four SARS-CoV-2 main protease (M) inhibitors including boceprevir, calpain inhibitors II and XII, and GC-376 with potent antiviral activity against infectious SARS-CoV-2 in cell culture. In this study, we further characterized the mechanism of action of these four compounds using the SARS-CoV-2 pseudovirus neutralization assay. It was found that GC-376 and calpain inhibitors II and XII have a dual mechanism of action by inhibiting both viral M and host cathepsin L in Vero cells. To rule out the cell-type dependent effect, the antiviral activity of these four compounds against SARS-CoV-2 was also confirmed in type 2 transmembrane serine protease-expressing Caco-2 cells using the viral yield reduction assay. In addition, we found that these four compounds have broad-spectrum antiviral activity in inhibiting not only SARS-CoV-2 but also SARS-CoV, and MERS-CoV, as well as human coronaviruses (CoVs) 229E, OC43, and NL63. The mechanism of action is through targeting the viral M, which was supported by the thermal shift-binding assay and enzymatic fluorescence resonance energy transfer assay. We further showed that these four compounds have additive antiviral effect when combined with remdesivir. Altogether, these results suggest that boceprevir, calpain inhibitors II and XII, and GC-376 might be promising starting points for further development against existing human coronaviruses as well as future emerging CoVs.