The maturation process of high-affinity antibodies is a result of intricate interactions between B cells and follicular helper T (Tfh) cells occurring in lymphoid germinal centers. HIV infection induces significant chronic immune activation, phenotypic skewing, and inflammation driven by years of continuous viral replication. High levels of viremia as well as immune activation and dysfunction have been demonstrated to have a perturbing impact on the B cell memory compartment and contribute to B cell exhaustion. Counterintuitively, the factors associated with perturbation of the B cell compartment seem to be favorable for the generation of highly affinity-matured Env-specific antibodies in a minority of HIV-infected individuals. Thus, the impact of HIV antigenemia on B cells and Tfh cell interactions warrants further exploration. We therefore studied immunophenotypes of HIV-specific B cells in individuals with differing levels of viral control using HIV Env gp120 probes and characterized the functionality of matched T cells in peripheral blood. While CXCR5 CD4 T cells were significantly diminished in HIV progressors, we found that a small subset of gp120-specific interleukin-21 (IL-21)-secreting CXCR5 CD4 T cells were significantly associated with gp120-specific B cell frequencies. In contrast, neither bulk CXCR5 CD4 T cells nor other HIV antigen specificities were associated with gp120-specific B cell levels. HIV-specific B cells derived from elite controllers displayed greater amounts of gp120-specific B cells in the resting memory subset, whereas HIV-specific B cells in progressors accumulated in tissue-like and activated memory subsets. Furthermore, CXCR5 CD4 T cells from elite controllers showed a stronger capacity to induce B cell maturation and immunoglobulin class switching than cells from HIV progressors. Dissecting the factors that are involved in B cell maturation and antibody development is important for HIV vaccine design. In this study, we found that HIV Env-specific CXCR5 CD4 T cells that secrete interleukin-21 are strongly associated with B cell memory phenotypes and function. Moreover, we found that the immune responses of HIV controllers showed intrinsically better helper activity than those of HIV progressors.