Hormones, such as DHEA, cortisol, testosterone, and progesterone play an important part in the regulation of the human immune system. However, the exact role of endocrine factors in the production of antibodies, in this case SARS-CoV-2-specific antibodies, remains poorly understood. We investigated the association between hormone levels and SARS-CoV-2 spike-protein-specific IgG antibody titers in a large, diverse cohort of 861 vaccinated as well as vaccinated plus COVID-19 recovered individuals. We observed negative correlations between cortisol, progesterone, testosterone (in males), and SARS-CoV-2-specific antibody levels. In contrast, a positive correlation was found between DHEA and antibody titers in vaccinated males. These hormone-antibody relationships exhibited important sex-specific differences. Our findings demonstrate that hormonal factors are associated with modulating the antibody response to SARS-CoV-2, with implications for personalized approaches to vaccination and treatment. Furthermore, the wide variability in hormone levels within the healthy population also suggests the potential value of incorporating endocrine assessments into COVID-19 risk profiling. Further research is needed to fully elucidate the mechanistic underpinnings of these hormone-antibody relationships and explore their broader clinical applications in the context of the ongoing COVID-19 endemic.