Gilthead sea bream is a highly cultured marine fish throughout the Mediterranean area, but new and strict criteria of welfare are needed to assure that the intensification of production has no negative effects on animal farming. Most welfare indicators are specific to a given phase of the production cycle, but others such as the timing of puberty and/or sex reversal are of retrospective value. This is of particular relevance in the protandrous gilthead sea bream, in which the sex ratio is highly regulated at the nutritional level. Social and environmental factors (e.g., contaminant loads) also alter the sex ratio, but the contribution of the genetic component remains unclear. To assess this complex issue, five gilthead sea bream families representative of slow/intermediate/fast growth were grown out with control or a plant-based diet in a common garden system from early life to the completion of their sexual maturity in 3-year-old fish. The plant-based diet highly enhanced the male-to-female sex reversal. This occurred in parallel with the progressive impairment of growth performance, which was indicative of changes in nutrient requirements as the result of the different energy demands for growth and reproduction through development. The effect of a different nutritional and genetic background on the reproductive performance was also assessed by measurements of circulating levels of sex steroids during the two consecutive spawning seasons, varying plasma levels of 17β-estradiol (E) and 11-ketotestosterone (11-KT) with age, gender, diet, and genetic background. Principal component analysis (PCA) of 3-year-old fish displayed a gradual increase of the E/11-KT ratio from males to females with the improvement of nutritional/genetic background. Altogether, these results support the use of a reproductive tract scoring system for leading farmed fish toward their optimum welfare condition, contributing to improving the productivity of the current gilthead sea bream livestock.