Previous models suggest that indirect reciprocity (reputation) can stabilize large-scale human cooperation [K. Panchanathan, R. Boyd, , 499-502 (2004)]. The logic behind these models and experiments [J. Gross , , eadd8289 (2023) and O. P. Hauser, A. Hendriks, D. G. Rand, M. A. Nowak, , 36079 (2016)] is that a strategy in which individuals conditionally aid others based on their reputation for engaging in costly cooperative behavior serves as a punishment that incentivizes large-scale cooperation without the second-order free-rider problem. However, these models and experiments fail to account for individuals belonging to multiple groups with reputations that can be in conflict. Here, we extend these models such that individuals belong to a smaller, "local" group embedded within a larger, "global" group. This introduces competing strategies for conditionally aiding others based on their cooperative behavior in the local or global group. Our analyses reveal that the reputation for cooperation in the smaller local group can undermine cooperation in the larger global group, even when the theoretical maximum payoffs are higher in the larger global group. This model reveals that indirect reciprocity alone is insufficient for stabilizing large-scale human cooperation because cooperation at one scale can be considered defection at another. These results deepen the puzzle of large-scale human cooperation.