Contact tracing is a key component of successful management of COVID-19. Contacts of infected individuals are asked to quarantine, which can significantly slow down (or prevent) community spread. Contact tracing is particularly effective when infections are detected quickly, when contacts are traced with high probability, when the initial number of cases is low, and when social distancing and border restrictions are in place. However, the magnitude of the individual contribution of these factors in reducing epidemic spread and the impact of population immunity (due to either previous infection or vaccination), in determining contact tracing outputs is not fully understood. We present a delayed differential equation model to investigate how the immunity status and the relaxation of social distancing requirements affect contact tracing practices. We investigate how the minimal contact tracing efficiency required to keep an outbreak under control depends on the contact rate and on the proportion of immune individuals. Additionally, we consider how delays in outbreak detection and increased case importation rates affect the number of contacts to be traced daily. We show that in communities that have reached a certain immunity status, a lower contact tracing efficiency is required to avoid a major outbreak, and delayed outbreak detection and relaxation of border restrictions do not lead to a significantly higher risk of overwhelming contact tracing. We find that investing in testing programs, rather than increasing the contact tracing capacity, has a larger impact in determining whether an outbreak will be controllable. This is because early detection activates contact tracing, which will slow, and eventually reverse exponential growth, while the contact tracing capacity is a threshold that will easily become overwhelmed if exponential growth is not curbed. Finally, we evaluate quarantine effectiveness in relation to the immunity status of the population and for different viral variants. We show that quarantine effectiveness decreases with increasing proportion of immune individuals, and increases in the presence of more transmissible variants. These results suggest that a cost-effective approach is to establish different quarantine rules for immune and nonimmune individuals, where rules should depend on viral transmissibility after vaccination or infection. Altogether, our study provides quantitative information for contact tracing downsizing in vaccinated populations or in populations that have already experienced large community outbreaks, to guide COVID-19 exit strategies.