The development of biodegradable materials with high osteogenic bioactivity is important for achieving rapid bone regeneration. Although hydroxyapatite (HAp) has been applied as a biomaterial for bone engineering due to its good osteoconductivity, conventional synthetic HAp nanomaterials still lack sufficient osteogenesis, likely due to their high crystallinity and uncontrollable architecture. A design of HAp nanoparticles mimicking bone features may create good microenvironments that promote osteogenesis for rapid bone regeneration. In this study, HAp nanoparticles with a comparatively less crystalline structure and nanorod shapes mimicking biological HAp nanocrystals of natural bone were fabricated using a simple chemical precipitation approach with mild temperature control in the absence of any organic solvents. Transmission electron microscopy (TEM) indicated that HAp nanorods with aspect ratios from 2.0 to 4.4 were synthesized by adjusting the reaction time as well as the reaction temperature. Fourier transform infrared spectroscopy and X-ray diffraction experiments displayed that HAp nanorods prepared at 30 °C (HAp-30 with an aspect ratio of 2.9) had a low crystalline structure and B-type CO substitution similar to those of natural HAp originating from bone tissue. The energy-dispersive spectroscopy (EDS) results showed that the Ca/P ratio of HAp-30 was 1.66 ± 0.13. An in vitro biological evaluation against rat bone marrow-derived mesenchymal stem cells indicated that the resulting HAp nanorods had excellent biocompatibility (with an ∼80-fold increase in IC50 compared to that of conventional HAp nanoparticles). Interestingly, the alkaline phosphatase (ALP), alizarin red S, and immunofluorescence staining results all showed that stem cells display an obvious osteogenesis dependence on the HAp nanostructure. Specifically, HAp nanorods with a moderate aspect ratio had the optimal osteogenic capacity (e.g., HAp-30 offered a 2.8-fold increase in ALP expression and a 4-fold increase in OCN expression relative to that provided by irregular HAp at day 14). It is expected that HAp nanorods with controllable architectures and size have potential as a kind of new bioactive bone filler for bone defect repair.