Cadmium exposure has been associated with adverse perinatal outcomes. One possible mechanism is endocrine disruption. Studies of non-pregnant adults suggest that cadmium impacts androgen production; here, we examined these associations during pregnancy. Participants in the Understanding Pregnancy Signals and Infant Development (UPSIDE) cohort provided biospecimens and questionnaire data in each trimester (n = 272). We quantified urinary cadmium, serum total testosterone (TT), estrone, estradiol, and estriol and serum free testosterone (fT). In adjusted longitudinal models, we examined sex steroid concentrations across pregnancy in relation to specific gravity-adjusted, ln-transformed cadmium concentrations. Additionally, we examined trimester-specific associations and stratified models by fetal sex. Results are presented as percent change (%∆) in hormone concentrations. In longitudinal models, higher cadmium concentrations were associated with lower fT across pregnancy (%∆ = -5.19, 95%CI: -8.33, -1.93), with no differences in other hormones observed. In trimester-specific models, higher cadmium concentrations were associated with lower TT in trimester 2 (%∆ = -15.26, 95%CI: -25.15, -4.06) and lower fT in trimester 3 (%∆ = -14.35, 95%CI: -19.75, -8.59). Associations with TT were stronger in pregnancies carrying female fetuses. Maternal cadmium exposure may be associated with reduced testosterone in pregnancy. Additional work is necessary to understand how alterations in gestational testosterone activity may impact pregnancy and child health.