With unprecedented speed, multiple vaccines against SARS-CoV-2 are available 1 year after the COVID-19 pandemic was first identified. As we push to achieve global control through these new vaccines, old challenges present themselves, including cold-chain storage, the logistics of mass vaccination, and vaccine hesitancy. Understanding how much hesitancy toward COVID-19 vaccines might occur and what factors may be driving these concerns can improve the ability of public health workers and communicators to maximize vaccine uptake. We nested a survey within a measles-rubella mass vaccination campaign in Zambia in November 2020 and asked about sentiments and beliefs toward COVID-19 and COVID-19 vaccines. Among parents bringing their children to receive a measles-rubella vaccine, we found high acceptability of COVID-19 vaccination of their children, but substantial uncertainty and hesitancy about receiving the vaccine themselves. COVID-19 vaccination hesitancy was correlated with beliefs around COVID-19 severity and risk, as well as vaccine safety and effectiveness.