The COVID-19 pandemic caused by SARS-CoV-2 highlighted the importance of reliable detection methods for disease control and surveillance. Optimizing detection antibodies by rational screening antigens would improve the sensitivity and specificity of antibody-based detection methods such as colloidal gold immunochromatography. In this study, we screened three peptide antigens with conserved sequences in the N protein of SARS-CoV-2 using bioinformatical and structural biological analyses. Antibodies that specifically recognize these peptides were prepared. The epitope of the peptide that had the highest binding affinity with its antibody was located on the surface of the N protein, which was favorable for antibody binding. Using the optimal antibody that can recognize this epitope, we developed colloidal gold immunochromatography, which can detect the N protein at 10 pg/mL. Importantly, this antibody could effectively recognize both the natural peptide antigen and mutated peptide antigen in the N protein, showing the feasibility of being applied in the large-scale population testing of SARS-CoV-2. Our study provides a platform with reference significance for the rational screening of detection antibodies with high sensitivity, specificity, and reliability for SARS-CoV-2 and other pathogens.