Mapping the melatonin receptor. 3. Design and synthesis of melatonin agonists and antagonists derived from 2-phenyltryptamines.

Three series of 2-phenyltryptamides were prepared as melatonin analogues to investigate the nature of the binding site of the melatonin receptor in chicken brain and in Xenopus laevis melanophore cells. The 5-methoxy-2-phenyltryptamides (6a-j) have high binding affinities for the chicken brain receptor, in some cases (6a-d) greater than that for melatonin, confirming and extending the work of Spadoni et al., and act as agonists in the Xenopus melanophore assay. Analogues lacking the 5-methoxyl group (2a-n) had a considerably lower affinity for the chicken brain receptor. In the Xenopus melanophore assay the compounds acylated on nitrogen by an alkyl group (2a-d) were agonists whereas the compounds acylated on nitrogen by an alicyclic group (2f-i) were antagonists. Introducing a methyl group at N1 (7) led to an increase in binding affinity in the chicken brain assay, whereas introducing an ethyl group (13) led to a decrease in binding affinity. A methyl substituent at the beta-position of the 3-amidoethane side chain (8, 11) also led to an increase in the binding affinity. The only analogue acylated on nitrogen with an alkyl group (acetyl) which showed antagonist activity was 9, which has a beta-methoxymethyl side chain. In the absence of the 5-methoxyl group the methoxymethyl function may cause the molecule to bind in a different configuration so that it is no longer able to activate the receptor. All of these observations are in agreement with a model of melatonin at the receptor site in which the 3-amidoethane side chain is in a conformation close to the 5-methoxyl group.