This cross-sectional exploratory study aims to verify associations between condom-protected sex, condom negotiation self-efficacy, self-esteem, and four minority stressors (experiences with misgendering, "passing" concerns, anticipated prejudice, and perceived prejudice) among transgender men (TM) and transgender women (TW). 260 individuals (192 TW and 68 TM) residing in two Brazilian states participated in the study. Data was collected online and in two hospital programs for transgender people and included sociodemographic data, condom-protected sex, the Trans-Specific Condom/Barrier Negotiation Self-Efficacy (T-Barrier) Scale, the Rosenberg Self-Esteem Scale, and four minority stressors. Measures that were significantly associated with condom-protected sex were tested as independent variables in a linear regression model. The main results suggest that lower condom negotiation self-efficacy, higher "passing" concerns, and higher experiences with misgendering were predictors of lower frequency of condom-protected sex. These negative outcomes were found among both TM and TW, which justifies their inclusion in public health policies. Structural strategies and clinical interventions are suggested to address condom negotiation self-efficacy and "passing" concerns in transgender populations.