While vaccines traditionally have been designed and used for protection against infection or disease caused by one specific pathogen, there are known off-target effects from vaccines that can impact infection from unrelated pathogens. The best-known non-specific effects from an unrelated or heterologous vaccine are from the use of the Bacillus Calmette-Guérin (BCG) vaccine, mediated partly through trained immunity. Other vaccines have similar heterologous effects. This review covers molecular mechanisms behind the heterologous effects, and the potential use of heterologous vaccination in the current COVID-19 pandemic. We then discuss novel pandemic response strategies based on rapidly deployed, widespread heterologous vaccination to boost population-level immunity for initial, partial protection against infection and/or clinical disease, while specific vaccines are developed.