We examined how HIV-related self-stigma was associated with different domains of quality of life (QoL), as measured by the World Health Organization Quality of Life in HIV-infected persons instrument (WHOQOL-HIV-Bref), and health-related quality of life (HRQoL) as measured by the generic 15D (15-dimensional measure of HRQoL), to identify the factors associated with self-stigma of people living with HIV (PLWH). The study sample included 440 patients living with HIV followed at the Infectious Disease Clinic of Helsinki University Hospital. Participants with more severe self-stigma reported significantly lower QoL and HRQoL. Male gender, cohabiting with a partner, and disclosure of HIV status were associated with less self-stigma; high education level and financial difficulties were associated with greater self-stigma. Having lived longer with HIV, being unemployed, and living alone were also predictors of self-stigma via financial difficulties. The findings suggest that self-stigma is a complex and multidimensional phenomenon that impacts the HRQoL of PLWH. Psychosocial interventions to enhance the well-being of PLWH are increasingly needed.