Listeria monocytogenes is a facultative intracellular bacterial pathogen that escapes from a phagosome and grows in the host cell cytosol. Escape of the bacterium from the phagosome to the cytosol is mediated by the bacterial pore-forming protein listeriolysin O (LLO). LLO has multiple mechanisms that optimize activity in the phagosome and minimize activity in the host cytosol. Mutants that fail to compartmentalize LLO activity are cytotoxic and have reduced virulence. We sought to determine why cytotoxic bacteria have attenuated virulence in the mouse model of listeriosis. In this study, we constructed a series of strains with mutations in LLO and with various degrees of cytotoxicity. We found that the more cytotoxic the strain in cell culture, the less virulent it was in mice. Induction of neutropenia increased the relative virulence of the cytotoxic strains 100-fold in the spleen and 10-fold in the liver. The virulence defect was partially restored in neutropenic mice by adding gentamicin, an antibiotic that kills extracellular bacteria. Additionally, L. monocytogenes grew more slowly in extracellular fluid (mouse serum) than within tissue culture cells. We concluded that L. monocytogenes controls the cytolytic activity of LLO to maintain its nutritionally rich intracellular niche and avoid extracellular defenses of the host.