Immunotherapy has emerged as an effective and life-changing approach for several types of cancers, both liquid and solid tumors. In combination with traditional treatments such as radiotherapy and/or chemotherapy, immune checkpoints inhibitors have improved prognosis and overall survival of patients with advanced melanoma and many other cancers. Among adoptive cell therapies (ACT), while chimeric antigen receptor T cell therapies have demonstrated remarkable efficacy in some hematologic malignancies, such as B cell leukemias, their success in solid tumors remains scarce due to the characteristics of the tumor microenvironment. On the other hand, ACT using tumor-infiltrating lymphocytes (TILs) is arguably the most effective treatment for metastatic melanoma patients, but even if their isolation has been achieved in epithelial tumors, their success beyond melanoma remains limited. Here, we review several aspects impacting TIL- and gene-modified "" TIL-based therapies and discuss future challenges that must be addressed with these approaches.