Chimeric antigen receptor (CAR) cell therapy offers patients with hematological malignancies a new therapeutic option. Traditionally, autologous T cells are used to generate CAR designed T cells for each patient. However, this method has several drawbacks, the development of allogeneic CAR cell therapy would be a promising breakthrough that could address several of these limitations. From the clinical trials that have published data, the efficacy of allogeneic CAR cell therapy did not meet the expectations. Because of the host-versus-graft (HvG) effect, allogeneic CAR cells are eliminated by the host, resulting in short-term persistence of allogeneic CAR cells and poor efficacy. It is critical to solve the HvG effect of allogeneic CAR cells. The current commonly used methods are suppressing the host's immune system, using HLA-matched homozygous donors, reducing the expression of HLA, targeting alloreactive lymphocytes and eliminating anti-CAR activities. In this review, we will focus on the HvG effect of the "off-the-shelf" allogeneic CAR cell therapy, especially its mechanism and current methods to solve this problem and summarize relevant clinical trial data.