SARS-CoV-2 mRNA vaccination induces robust humoral and cellular immunity in the circulation; however, it is currently unknown whether it elicits effective immune responses in the respiratory tract, particularly against variants of concern (VOCs), including Omicron. We compared the SARS-CoV-2 S-specific total and neutralizing antibody responses, and B and T cell immunity, in the bronchoalveolar lavage fluid (BAL) and blood of COVID-19-vaccinated individuals and hospitalized patients. Vaccinated individuals had significantly lower levels of neutralizing antibody against D614G, Delta (B.1.617.2), and Omicron BA.1.1 in the BAL compared with COVID-19 convalescents despite robust S-specific antibody responses in the blood. Furthermore, mRNA vaccination induced circulating S-specific B and T cell immunity, but in contrast to COVID-19 convalescents, these responses were absent in the BAL of vaccinated individuals. Using a mouse immunization model, we demonstrated that systemic mRNA vaccination alone induced weak respiratory mucosal neutralizing antibody responses, especially against SARS-CoV-2 Omicron BA.1.1 in mice; however, a combination of systemic mRNA vaccination plus mucosal adenovirus-S immunization induced strong neutralizing antibody responses not only against the ancestral virus but also the Omicron BA.1.1 variant. Together, our study supports the contention that the current COVID-19 vaccines are highly effective against severe disease development, likely through recruiting circulating B and T cell responses during reinfection, but offer limited protection against breakthrough infection, especially by the Omicron sublineage. Hence, mucosal booster vaccination is needed to establish robust sterilizing immunity in the respiratory tract against SARS-CoV-2, including infection by the Omicron sublineage and future VOCs.