Neuropsychiatric disorders show shared and distinct neurobiological correlates. A cross-disorder genome-wide association study (GWAS) identified 23 highly pleiotropic single-nucleotide polymorphisms (SNPs) that were associated with at least four neuropsychiatric disorders, and 22 SNPs that were associated predominantly with schizophrenia. Exploring their link to brain-related traits might advance understanding their distinct neurobiological processes. Using the UK Biobank data (n = 28,952), this study examined the association of both a genetic risk score (GRS) for highly pleiotropic SNPs (PleioPsych-GRS), and a GRS for predominantly schizophrenia-associated SNPs (SCZ-GRS) with 154 measures of subcortical volume, cortical thickness, and surface area as well as 12 outcomes related to mental health. To generate further insights at the individual SNP level, the association between SNPs and brain structure was examined using GWAS summary statistics. The PleioPsych-GRS showed no significant association with brain structure after correction for multiple testing. The SCZ-GRS showed a significant association with an increased surface area of the lateral orbitofrontal region, and an increased volume of the putamen, among others. The PleioPsych-GRS and the SCZ-GRS were associated with eight and four outcomes related to mental health, respectively. Two highly pleiotropic and 10 SCZ-associated SNPs were associated with several structural brain phenotypes. Taken together, these findings indicated that GRSs of highly pleiotropic SNPs and predominantly schizophrenia-associated SNPs have partly distinct associations with brain structure and outcomes related to mental health. Thus, investigating schizophrenia-specific and pleiotropic variants may improve our understanding of the neurobiology of neuropsychiatric disorders.