Over the last several years, significant progress has been made toward understanding the mechanisms involved in the mineralization of hard collagenous tissues, such as bone and dentin. Particularly notable are the identification of transient mineral phases that are precursors to carbonated hydroxyapatite, the identification and characterization of non-collagenous proteins that are involved in controlling mineralization, and significant improvements in our understanding of the structure of collagen. These advances not only represent a paradigm shift in the way collagen mineralization is viewed and understood, but have also brought new challenges to light. In this review, we discuss how recent in vitro models have addressed critical questions regarding the role of the non-collagenous proteins in controlling mineralization, the nature of the interactions between amorphous calcium phosphate and collagen during the early stages of mineralization, and the role of collagen in the mineralization process. We discuss the significance of these findings in expanding our understanding of collagen biomineralization, while addressing some of the limitations that are inherent to in vitro systems.