While competitive training is usually associated with the prevalence of back pain and injuries in athletes, little attention is being paid to the positive effects of sport-specific exercises on core musculature in the prevention of back problems. This scoping review aims (i) to map the literature that addresses the effects on reduction of back problems following athlete training with differing demands on the core musculature and (ii) to identify gaps in the existing literature and propose future research on this topic. The main literature search was conducted on the MEDLINE, PubMed, Web of Science, Scopus, and Cochrane Library databases and was completed on Elsevier, SpringerLink, and Google Scholar. A total of 21 research articles met the inclusion criteria. The findings of 17 studies identified that core strengthening and core stabilization exercises, alone or in combination with athlete training, contribute to the reduction of back pain in athletes, whereas only four studies revealed no significant association of core muscle strength and/or endurance with back problems. Nevertheless, more research is warranted to elucidate the pros and cons of purely sport-specific training with differing demands on the core musculature on back health in athletes. This could help us to design prevention strategies specifically tailored to individual athletes.