An immunocytochemical and autoradiographic investigation of the serotoninergic innervation of trigeminal mesencephalic and motor nuclei in the rabbit.

The results of a previous experiment suggest that the cell bodies of many jaw closing muscle spindle afferents in the trigeminal mesencephalic nucleus of the rabbit are phasically inhibited during fictive mastication. The aim of this study was to investigate one possible neurotransmitter system that could be involved in this modulation, serotonin, by use of receptor autoradiography techniques and immunofluorescence combined with retrograde labelling of masseteric spindle afferents and motoneurons. A second objective was to compare the serotonin innervation of neurons in the trigeminal mesencephalic nucleus with that of masseteric motoneurons. Serotoninergic fibres were seen surrounding labelled masseteric spindle afferents, as well as unlabelled neurons, in the trigeminal mesencephalic nucleus. These fibres were close to the cell bodies and sometimes to the axon hillocks of the neurons. Although it has been reported that many neurons of the trigeminal nucleus are multipolar in some species, none of the labelled spindle afferent in this study had more than one process. Throughout the motor trigeminal nucleus, serotonin fibres were found in close proximity with cell bodies and with the proximal portions of axons and dendrites of labelled and unlabelled motoneurons. Serotonin fibres were also seen adjacent to cell bodies and processes of efferent neurons in cell group k. Autoradiography with several tritiated ligands was used to reveal the presence of receptors for serotonin as well as its uptake sites. Only serotonin2 receptors were found to be abundant in the trigeminal mesencephalic nucleus. The motor nucleus and cell group k contained serotonin2 and serotonin3 receptors, as well as serotonin uptake sites. Serotonin1A receptors appear to be absent from both nuclei. The findings suggest that release of serotonin from fibres in close proximity to trigeminal primary afferent somata could modify the transmission of action potentials from muscle spindle receptors during mastication through an action on serotonin2 receptors. In the motor nucleus and cell group k, serotonin may alter neuronal properties through actions on at least two receptor subtypes (serotonin2 and serotonin3).