Recent burst of new technologies that allow for quantitatively delineating chromatin structure has greatly expanded our understanding of how the genome is organized in the three-dimensional (3D) space of the nucleus. It is now clear that the hierarchical organization of the eukaryotic genome critically impacts nuclear activities such as transcription, replication, as well as cellular and developmental events such as cell cycle, cell fate decision and embryonic development. In this review, we discuss new insights into how the structural features of the 3D genome hierarchy are established and maintained, how this hierarchy undergoes dynamic rearrangement during normal development and how its perturbation will lead to human disease, highlighting the accumulating evidence that links the diverse 3D genome architecture components to a multitude of human diseases and the emerging mechanisms by which 3D genome derangement causes disease phenotypes.