Introducing nonlinear fluorophore excitation into structured illumination microscopy (SIM) can further extend its spatial resolution without theoretical limitation. However, it is a great challenge to recover the weak higher-order harmonic signal and reconstruct high-fidelity super-resolution (SR) images. Here, we proposed a joint optimization strategy in both the frequency and spatial domains to reconstruct high-quality nonlinear SIM (NL-SIM) images. We demonstrate that our method can reconstruct SR images with fewer artifacts and higher fidelity on the BioSR dataset with patterned-activation NL-SIM. This method could robustly overcome one of the long-lived obstacles on NL-SIM imaging, thereby promoting its wide application in biology.