Risk and protective factors for psychiatric illnesses are linked to distinct structural and functional changes in the brain. Further, the prevalence of these factors varies across sexes and genders, yet the distinct and joint effects of sex and gender in this context have not been extensively characterized. This suggests that risk and protective factors may map onto the brain and uniquely influence individuals across sexes and genders. Here, we review how specific risk (childhood maltreatment, the COVID-19 pandemic, experiences of racism), and protective factors (social support and psychological resilience) distinctly influence the brain across sexes and genders. We also discuss the role of sex and gender in the compounding effects of risk factors and in the interdependent influences of risk and protective factors. As such, we call on researchers to consider sex and gender when researching risk and protective factors for psychiatric illnesses, and we provide concrete recommendations on how to account for them in future research. Considering protective factors alongside risk factors in research and acknowledging sex and gender differences will enable us to establish sex- and gender-specific brain-behavior relationships. This will subsequently inform the development of targeted prevention and intervention strategies for psychiatric illnesses, which have been lacking. To achieve sex and gender equality in mental health, acknowledging and researching potential differences will lead to a better understanding of men and women, males and females, and the factors that make them more vulnerable or resilient to psychopathology.