In contrast to findings from cohorts comprised primarily of HIV-infected men, verbal memory deficits are the largest cognitive deficit found in HIV-infected women from the Women's Interagency HIV Study (WIHS), and this deficit is not explained by depressive symptoms or substance abuse. HIV-infected women may be at greater risk for verbal memory deficits due to a higher prevalence of cognitive risk factors such as high psychosocial stress and lower socioeconomic status. Here, we investigate the association between perceived stress using the Perceived Stress Scale (PSS-10) and verbal memory performance using the Hopkins Verbal Learning Test (HVLT) in 1009 HIV-infected and 496 at-risk HIV-uninfected WIHS participants. Participants completed a comprehensive neuropsychological test battery which yielded seven cognitive domain scores, including a primary outcome of verbal memory. HIV infection was not associated with a higher prevalence of high perceived stress (i.e., PSS-10 score in the top tertile) but was associated with worse performance on verbal learning (p < 0.01) and memory (p < 0.001), as well as attention (p = 0.02). Regardless of HIV status, high stress was associated with poorer performance in those cognitive domains (p's < 0.05) as well as processing speed (p = 0.01) and executive function (p < 0.01). A significant HIV by stress interaction was found only for the verbal memory domain (p = 0.02); among HIV-infected women only, high stress was associated with lower performance (p's < 0.001). That association was driven by the delayed verbal memory measure in particular. These findings suggest that high levels of perceived stress contribute to the deficits in verbal memory observed in WIHS women.