Pain is a global health problem that leads to sedentary behavior and tends to cause negative emotion. In contrast, exercise is widely recommended for a health promotion, while pain often worsens with physical activity. Although exercise therapy is often prescribed to people with pain, the mechanisms of exercise effect on pain remains unclear. In this study, we tried to identify a universal association factor between regular exercise and pain intensity utilizing a cross-sectional web-based survey involving 52,353 adult participants from a large national study conducted in Japan. Using principal component analysis, we uncovered a mediation model of exercise effect on pain through psychological components. Analyses were performed in half of the population with pain (n = 20,330) and validated in the other half (n = 20,330), and showed that high-frequency exercise had a significant association with reduction in pain intensity. We also found Negative Affect and Vigor, two psychological components, are fully associating the exercise effect on pain (indirect effect =  - 0.032, p < 0.001; association proportion = 0.99) with a dose-dependent response corresponding to the frequency of exercise. These findings were successfully validated (indirect effect of high-frequency exercise =  - 0.028, p < 0.001; association proportion = 0.85). Moreover, these findings were also identified in subpopulation analyses of people with low back, neck, knee pain, and the tendency of the exercise effect on pain was increased with older people. In conclusion, the effect of exercise on pain is associated with psychological components and these association effects increased in parallel with the frequency of exercise habit regardless pain location.