Examining the role of muscarinic M5 receptors in VTA cholinergic modulation of depressive-like and anxiety-related behaviors in rats.

Acetylcholine is implicated in mood disorders including depression and anxiety. Increased cholinergic tone in humans and rodents produces pro-depressive and anxiogenic-like effects. Cholinergic receptors in the ventral tegmental area (VTA) are known to mediate these responses in male rats, as measured by the sucrose preference test (SPT), elevated plus maze (EPM), and the forced swim test (FST). However, these effects have not been examined in females, and the VTA muscarinic receptor subtype(s) mediating the pro-depressive and anxiogenic-like behavioral effects of increased cholinergic tone are unknown. We first examined the behavioral effects of increased VTA cholinergic tone in male and female rats, and then determined whether VTA muscarinic M5 receptors were mediating these effects. VTA infusion of the acetylcholinesterase inhibitor physostigmine (0.5 μg, 1 μg and 2 μg/side) in males and females produced anhedonic-like, anxiogenic, pro-depressive-like responses on the SPT, EPM, and FST. In females, VTA administration of the muscarinic M5 selective negative allosteric modulator VU6000181 (0.68 ng, 2.3 ng, 6.8 ng/side for a 3 μM, 10 μM, 30 μM/side infusion) did not alter SPT, EPM nor FST behavior. However, in males intra-VTA infusion of VU6000181 alone reduced time spent immobile on the FST. Furthermore, co-infusion of VU6000181 with physostigmine, in male and female rats, attenuated the pro-depressive and anxiogenic-like behavioral responses induced by VTA physostigmine alone, in the SPT, EPM, and FST. Together, these data reveal a critical role of VTA M5 receptors in mediating the anhedonic, anxiogenic, and depressive-like behavioral effects of increased cholinergic tone in the VTA.