Bone remodeling is energetically demanding process. Energy coming from nutrients present in the diet contributes to function of different cell type including osteoblasts, osteocytes and osteoclasts in bone marrow participating in bone homeostasis. With aging, obesity and osteoporosis the function of key building blocks, bone marrow stromal cells (BMSCs), changes towards higher accumulation of bone marrow adipose tissue (BMAT) and decreased bone mass, which is affected by diet and sex dimorphism. Men and women have unique nutritional needs based on physiological and hormonal changes across the life span. However, the exact molecular mechanisms behind these pathophysiological conditions in bone are not well-known. In this review, we focus on bone and BMAT physiology in men and women and how this approach has been taken by animal studies. Furthermore, we discuss the different diet interventions and impact on bone and BMAT in respect to sex differences. We also discuss the future perspective on precision nutrition with a consideration of sex-based differences which could bring better understanding of the diet intervention in bone health and weight management.