Our understanding of the physical principles organizing the genome in the nucleus is limited by the lack of tools to directly exert and measure forces on interphase chromosomes in vivo and probe their material nature. Here, we introduce an approach to actively manipulate a genomic locus using controlled magnetic forces inside the nucleus of a living human cell. We observed viscoelastic displacements over micrometers within minutes in response to near-piconewton forces, which are consistent with a Rouse polymer model. Our results highlight the fluidity of chromatin, with a moderate contribution of the surrounding material, revealing minor roles for cross-links and topological effects and challenging the view that interphase chromatin is a gel-like material. Our technology opens avenues for future research in areas from chromosome mechanics to genome functions.