Anti-idiotypic antibodies (Ab2) are valuable tools that can be used for a better understanding of molecular mimicry and the immunological network. In this work, we showed a new application of a phage-displayed alpha-type Ab2 (Ab2α) to improve the sensitivity of an enzyme-linked immunosorbent assay (ELISA) detecting cyanobacterial toxin microcystin-LR (MC-LR). A monoclonal antibody (mAb) against MC-LR was used as an antigen to isolate binders in a camelid nanobody library. After three rounds of panning, three unique clones with strong binding against anti-MC-LR mAbs were isolated. These clones could specifically bind to anti-MC-LR mAbs without influencing mAbs binding with MC-LR, meaning these clones were Ab2αs. Based on the signal amplification effect of phage coat proteins and the non-competitive nature of Ab2α, a novel competitive ELISA method for MC-LR was established with a phage-displayed Ab2α. It showed that the phage-displayed Ab2α greatly enhanced the ELISA signal and sensitivity of the method was improved 3.5-fold to the conventional one. Combining with the optimization of pre-incubation time, the optimized ELISA decreased its limit of detection (LOD) from 4.5 ng/mL to 0.8 ng/mL (5.6-fold improvement). This new application of Ab2α may potentially be employed to improve the sensitivity of immunoassays for other environmental pollutants.