To mitigate the spread of SARS-CoV-2, the Swiss government enacted restrictions on social contacts from 2020 to 2022. In addition, individuals changed their social contact behavior to limit the risk of COVID-19. In this study, we aimed to investigate the changes in social contact patterns of the Swiss population. As part of the CoMix study, we conducted a survey consisting of 24 survey waves from January 2021 to May 2022. We collected data on social contacts and constructed contact matrices for the age groups 0-4, 5-14, 15-29, 30-64, and 65 years and older. We estimated the change in contact numbers during the COVID-19 pandemic to a synthetic pre-pandemic contact matrix. We also investigated the association of the largest eigenvalue of the social contact and transmission matrices with the stringency of pandemic measures, the effective reproduction number (R), and vaccination uptake. During the pandemic period, 7084 responders reported an average number of 4.5 contacts (95% confidence interval, CI: 4.5-4.6) per day overall, which varied by age and survey wave. Children aged 5-14 years had the highest number of contacts with 8.5 (95% CI: 8.1-8.9) contacts on average per day and participants that were 65 years and older reported the fewest (3.4, 95% CI: 3.2-3.5) per day. Compared with the pre-pandemic baseline, we found that the 15-29 and 30-64 year olds had the largest reduction in contacts. We did not find statistically significant associations between the largest eigenvalue of the social contact and transmission matrices and the stringency of measures, R, or vaccination uptake. The number of social contacts in Switzerland fell during the COVID-19 pandemic and remained below pre-pandemic levels after contact restrictions were lifted. The collected social contact data will be critical in informing modeling studies on the transmission of respiratory infections in Switzerland and to guide pandemic preparedness efforts.