A dual-mode immunoassay method was developed for colorimetric and fluorescence detection of aflatoxin B1 (AFB1) based on streptavidin-induced gold nanoparticle aggregation (AuNP@SA). AuNP-modified streptavidin-biotin labeling AFB1 complete antigen aggregations (AuNP@SA@Bio-BSA-AFB1) were synthesized as the competitive binding and dual-mode probe. AuNP@SA@Bio-BSA-AFB1 aggregations possessed high colorimetric and fluorescence quenching intensities. AFB1 antibodies modified immunomagnetic microspheres were used as the capture probe. The competitive binding between AFB1 and AuNP@SA@Bio-BSA-AFB1 leads to changes in color and fluorescence intensity. The detection limit of the colorimetric method is 6.95 ng·mL, while that of the fluorescence method is 0.07 ng·mL. The practicality of the proposed strategy was demonstrated by determining AFB1 in spiked peanut samples.