Bone is a composite material in which collagen fibrils form a scaffold for a highly organized arrangement of uniaxially oriented apatite crystals. In the periodic 67 nm cross-striated pattern of the collagen fibril, the less dense 40-nm-long gap zone has been implicated as the place where apatite crystals nucleate from an amorphous phase, and subsequently grow. This process is believed to be directed by highly acidic non-collagenous proteins; however, the role of the collagen matrix during bone apatite mineralization remains unknown. Here, combining nanometre-scale resolution cryogenic transmission electron microscopy and cryogenic electron tomography with molecular modelling, we show that collagen functions in synergy with inhibitors of hydroxyapatite nucleation to actively control mineralization. The positive net charge close to the C-terminal end of the collagen molecules promotes the infiltration of the fibrils with amorphous calcium phosphate (ACP). Furthermore, the clusters of charged amino acids, both in gap and overlap regions, form nucleation sites controlling the conversion of ACP into a parallel array of oriented apatite crystals. We developed a model describing the mechanisms through which the structure, supramolecular assembly and charge distribution of collagen can control mineralization in the presence of inhibitors of hydroxyapatite nucleation.