Air pollution increases the risks of all-cause mortality, cardiovascular mortality and respiratory mortality across China. However, the urban-rural differences in the associations between air pollution and mortality have not been clearly identified. In this study, a distributed lag nonlinear model was used to examine whether the air pollutants-mortality associations vary between urban and rural areas. Then, we used logistic regression analyses to evaluate the air pollutants-mortality relations. Also, generalized additive models were simulated to evaluate the nonlinear curves. Our results showed that the relative risks of air pollution-related mortality were generally higher in rural areas, where PM pollution was the dominant factor (p-value < 0.05). Mortality risks for all-cause, cardiovascular and respiratory will increase when average annual PM concentrations exceed approximately 38 μg/m, 41 μg/m and 41 μg/m, respectively, all of which exceed the annual Grade II standards. In urban areas, PM and NO were associated with mortality (p-value < 0.05). We proposed some area-specific strategies for controlling the NO pollution and PM pollution in urban areas and the PM pollution in rural areas to eliminate the gaps. Our findings identify that rural residents are more sensitive to air pollution than urban residents in China, and this result challenges previous assumptions about the more adverse effects of urbanization on residents' health in developing countries.