Cellular interactions in the tumor microenvironment (TME) significantly govern cancer progression and drug response. The efficacy of clinical immunotherapies has fostered an exponential interest in the tumor immune microenvironment, which in turn has engendered a pressing need for robust experimental systems modeling patient-specific tumor-immune interactions. Traditional 2D in vitro tumor immunotherapy models have reconstituted immortalized cancer cell lines with immune components, often from peripheral blood. However, newly developed 3D in vitro organoid culture methods now allow the routine culture of primary human tumor biopsies and increasingly incorporate immune components. Here, we present a viewpoint on recent advances, and propose translational applications of tumor organoids for immuno-oncology research, immunotherapy modeling, and precision medicine.