A behavioral mechanistic investigation of the role of 5-HT receptors in the mediation of rat maternal behavior.

Previous work suggests that 5-HT receptors play a special role in rodent maternal aggression, but not in other aspects of maternal care (e.g. pup retrieval and nest building). The present study re-assessed the basic effects of 5-HT activation or blockade on various maternal responses in postpartum female rats. We also examined the possible behavioral mechanisms underlying the maternal effects of 5-HT. Sprague-Dawley mother rats were injected with a 5-HT agonist 8-OH-DPAT (0.1, 0.5 or 1.0 mg/kg, sc), a 5-HT antagonist WAY-101405 (0.1, 0.5 or 1.0 mg/kg, sc) or 0.9% saline solution on postpartum days 3, 5, and 7. Maternal behavior was tested 30 min before, 30 min, 120 min, and 240 min after the injection. Acute and repeated 8-OH-DPAT treatment significantly disrupted pup retrieval, pup licking, nursing, and nest building in a dose-dependent fashion, whereas WAY-101405 had no effect at the tested doses. The 5-HT receptor specificity of 8-OH-DPAT's action was confirmed as its maternal disruption effect was reversed by pretreatment of WAY-100635 (a highly selective 5-HT receptor antagonist). Subsequent pup preference test found that 8-OH-DPAT did not decrease the pup preference over a novel object, thus no inhibition on maternal motivation or maternal affect. The pup separation test and pup retrieval on an elevated plus maze test also failed to find any motivational and motor impairment effect with 8-OH-DPAT. However, 8-OH-DPAT at the maternal disruptive dose did disrupt the prepulse inhibition (a measure of attentional function) of acoustic startle response and enhanced the basal startle response. These findings suggest that stimulation of 5-HT receptors by 8-OH-DPAT impairs maternal care by partially interfering with the attentional processing or basal anxiety. More work is needed to further delineate the psychological and neuronal mechanisms underlying the maternal disruptive effect of 5-HT receptor activation.