Homotypic chromatin interactions and loop extrusion are thought to be the two main drivers of mammalian chromosome folding. Here we tested the role of RNA polymerase II (RNAPII) across different scales of interphase chromatin organization in a cellular system allowing for its rapid, auxin-mediated degradation. We combined Micro-C and computational modeling to characterize subsets of loops differentially gained or lost upon RNAPII depletion. Gained loops, extrusion of which was antagonized by RNAPII, almost invariably formed by engaging new or rewired CTCF anchors. Lost loops selectively affected contacts between enhancers and promoters anchored by RNAPII, explaining the repression of most genes. Surprisingly, promoter-promoter interactions remained essentially unaffected by polymerase depletion, and cohesin occupancy was sustained. Together, our findings reconcile the role of RNAPII in transcription with its direct involvement in setting-up regulatory three-dimensional chromatin contacts genome wide, while also revealing an impact on cohesin loop extrusion.