To investigate the dynamics of centromere organization, we have assessed the exchange rates of inner centromere proteins (CENPs) by quantitative microscopy throughout the cell cycle in human cells. CENP-A and CENP-I are stable centromere components that are incorporated into centromeres via a "loading-only" mechanism in G1 and S phase, respectively. A subfraction of CENP-H also stays stably bound to centromeres. In contrast, CENP-B, CENP-C, and some CENP-H and hMis12 exhibit distinct and cell cycle-specific centromere binding stabilities, with residence times ranging from seconds to hours. CENP-C and CENP-H are immobilized at centromeres specifically during replication. In mitosis, all inner CENPs become completely immobilized. CENPs are highly mobile throughout bulk chromatin, which is consistent with a binding-diffusion behavior as the mechanism to scan for vacant high-affinity binding sites at centromeres. Our data reveal a wide range of cell cycle-specific assembly plasticity of the centromere that provides both stability through sustained binding of some components and flexibility through dynamic exchange of other components.