Dual role of serotonin in the acquisition and extinction of reward-driven learning: involvement of 5-HT1A, 5-HT2A and 5-HT3 receptors.

Serotonin (5-HT) has been proposed as a possible encoder of reward. Nevertheless, the role of this neurotransmitter in reward-based tasks is not well understood. Given that the major serotonergic circuit in the rat brain comprises the dorsal raphe nuclei and the medial prefrontal cortex (mPFC), and because the latter structure is involved in the control of complex behaviors and expresses 1A (5-HT1A), 2A (5-HT2A), and 3 (5-HT3) receptors, the aim was to study the role of 5-HT and of these receptors in the acquisition and extinction of a reward-dependent operant conditioning task. Long Evans rats were trained in an operant conditioning task while receiving fluoxetine (serotonin reuptake inhibitor, 10mg/kg), tianeptine (serotonin reuptake enhancer, 10mg/kg), buspirone (5-HT1A partial agonist, 10mg/kg), risperidone (5-HT2A antagonist, 1mg/kg), ondansetron (5-HT3 antagonist, 2mg/kg) or vehicle. Then, animals that acquired the operant conditioning without any treatment were trained to extinct the task in the presence of the pharmacological agents. Fluoxetine impaired acquisition but improved extinction. Tianeptine administration induced the opposite effects. Buspirone induced a mild deficit in acquisition and had no effects during the extinction phase. Risperidone administration resulted in learning deficits during the acquisition phase, although it promoted improved extinction. Ondansetron treatment showed a deleterious effect in the acquisition phase and an overall improvement in the extinction phase. These data showed a differential role of 5-HT in the acquisition and extinction of an operant conditioning task, suggesting that it may have a dual function in reward encoding.