Studies have found physical inactivity to be a significant health risk factor and have demonstrated how physical inactivity behaviour varies according to social background. As a result, differences according to social background must be considered when examining motives and barriers related to physical activity and sport. This scoping review examines motives and barriers related to physical activity and sport among people with different social backgrounds, including age, socioeconomic status, gender, ethnic minority background and disability status. A systematic literature search was performed in four scientific databases and yielded 2935 articles of which 58 articles met the inclusion criteria. We identified common motives for physical activity and sport as health benefits, well-being, enjoyment, social interaction, and social support; common barriers as time restrictions, fatigue and lack of energy, financial restrictions, health-related restrictions, low motivation, and shortage of facilities. We also identified several motives and barriers that were specific to or more pronounced among people with different social backgrounds. The knowledge about motives and barriers related to physical activity and sport provided in this article can inform health promotion initiatives that seek to improve public health both in general and when specifically targeting groups of people with different social backgrounds.