Effects of conformationally restrained analogues of serotonin on its uptake and binding in rat brain.

Two series of serotonin analogues, in which the side chain amino group is constrained in the gauche or trans conformation, were utilized to study the preferred conformation of serotonin for interaction with two different neuronal sites. 6-Hydroxytetrahydro-beta-carboline and 6-hydroxy-3-aminotetrahydrocarbazole were found to be potent inhibitors of serotonin uptake into hypothalamic synaptosomes, with IC50 values of 0.13 microM for each analogue. The type of inhibition, as determined by Dixon plots, was found to be competitive, with Ki's of 3.0 X 10(-8) M and 4.6 X 10(-8) M for the beta-carboline and carbazole derivatives, respectively. Methoxylation or lack of a hydroxy group at the 6 position of the carbazole derivative did not alter inhibitory potency, while methoxy or benzyloxy substitution decreased potency 22- to 326-fold. The serotonin analogues were 20 to 30 times less potent in inhibiting the synaptosomal transport of the catecholamines. With regard to [3H]serotonin binding to membranes obtained from brain homogenates, both analogues exhibited poor affinity compared with the transmitter. However, the beta-carboline derivative was three times as potent as the carbazole analogue. These findings and earlier ones with regard to the effect of the serotonin analogues on brain monoamine oxidase activity support the idea that serotonin analogues interact differentially with the three different serotonergic sites examined.