Elderly people and patients with comorbidities are at higher risk of COVID-19 infection, resulting in severe complications and high mortality. However, the underlying mechanisms are unclear. In this study, we investigate whether miRNAs in serum exosomes can exert antiviral functions and affect the response to COVID-19 in the elderly and people with diabetes. First, we identified four miRNAs (miR-7-5p, miR-24-3p, miR-145-5p and miR-223-3p) through high-throughput sequencing and quantitative real-time PCR analysis, that are remarkably decreased in the elderly and diabetic groups. We further demonstrated that these miRNAs, either in the exosome or in the free form, can directly inhibit S protein expression and SARS-CoV-2 replication. Serum exosomes from young people can inhibit SARS-CoV-2 replication and S protein expression, while the inhibitory effect is markedly decreased in the elderly and diabetic patients. Moreover, three out of the four circulating miRNAs are significantly increased in the serum of healthy volunteers after 8-weeks' continuous physical exercise. Serum exosomes isolated from these volunteers also showed stronger inhibitory effects on S protein expression and SARS-CoV-2 replication. Our study demonstrates for the first time that circulating exosomal miRNAs can directly inhibit SARS-CoV-2 replication and may provide a possible explanation for the difference in response to COVID-19 between young people and the elderly or people with comorbidities.