The early attainment of high COVID-19 vaccination rates can minimize avoidable hospitalizations/deaths. The fifth wave COVID-19 outbreak in Hong Kong caused >9000 deaths, and most of them were unvaccinated older people. This study hence investigated determinants of taking the first dose vaccination at a later phase (Phase 3: during the fifth wave outbreak, i.e., February-July 2022) versus two earlier phases (Phase 1: first six months since vaccine rollout, i.e., February-July 2021; Phase 2: six months prior to the outbreak, i.e., August 2021-January 2022) via a random telephone survey among 386 ever-vaccinated Hong Kong older people aged ≥60 (June/July 2022). A total of 27.7%, 51.1%, and 21.3% took the first dose at Phase 1, Phase 2, and Phase 3, respectively. Unfavorable perceptions related to COVID-19/vaccination, exposure to conflicting/counter-information about the suitability of older people's vaccination from various sources, unsupportive family influences prior to the outbreak, and depressive symptoms were significantly associated with taking the first dose at Phase 3 instead of Phase 1 and Phase 2. To speed up COVID-19 vaccination and avoid unnecessary deaths, the government and health professionals should rectify misinformation, provide clear/consistent information for older people and their family members, and focus on those having depressive symptoms at an earlier stage of the pandemic.