Inspired by the natural extracellular matrix, the organic-inorganic composite nanofibers are promising scaffolds for bone tissue engineering. Chitosan-based nanofibers are widely used as bone tissue engineering scaffolds with good biocompatibility but pungent solvents are frequently used for its processing. Carboxymethyl chitosan (CMCS), a water-soluble derivative of chitosan, has better biodegradability and bioactivity which allows CMCS to chelate Ca and induce the deposition of apatite. Moreover, with water as solvent, CMCS nanofibers avoid the acidic salt removal comparing to electrospun-chitosan. In this study, we successfully prepared uniform CMCS nanofibers with the aid of polyethylene oxide (PEO) and obtained the optimized conditions with a voltage of 25 kV and PEO of molecular weight 1000 kDa. We further prepared hydroxyapatite (HA) coated electrospun CMCS nanofibers by biomimetic mineralization using 5 times simulated body fluid. The promotion of osteogenic differentiation of mouse bone marrow stromal cells (mBMSCs) in vitro was evaluated on the nanofibers scaffolds. Cell experiments revealed that CMCS-HA composite nanofibers increased the ALP activity. The gene expression level of Runx2 and ALP were about 1.6 and 4.3 folds at the 7 days, and 5.1 and 10 folds at the 14 days on CMCS-HA nanofibrous membranes than that on CMCS alone samples. The level of OCN increased by 24 and 1.5 times on the CMCS-HA scaffolds than CMCS scaffolds at the 14 and 21 days. In vivo new bone formation by nanofiber scaffolds was investigated in a critical-size rat calvarial bone defect model. Micro-CT results showed that the whole defect was covered by new bone after CMCS-HA filling the defect for 12 weeks. The results of H&E staining and Masson's trichrome staining on histological sections further confirmed that composite nanofibers promoted new bone formation and maturation.