Pancreatic Ductal Adenocarcinoma (PDAC) is projected to become the 2nd leading cause of cancer-related deaths in the United States. Limitations in early detection and treatment barriers contribute to the lack of substantial success in the treatment of this challenging-to-treat malignancy. Desmoplasia is the hallmark of PDAC microenvironment that creates a physical and immunologic barrier. Stromal support cells and immunomodulatory cells face aberrant signaling by pancreatic cancer cells that shifts the complex balance of proper repair mechanisms into a state of dysregulation. The product of this dysregulation is the desmoplastic environment that encases the malignant cells leading to a dense, hypoxic environment that promotes further tumorigenesis, provides innate systemic resistance, and suppresses anti-tumor immune invasion. This desmoplastic environment combined with the immunoregulatory events that allow it to persist serve as the primary focus of this review. The physical barrier and immune counterbalance in the tumor microenvironment (TME) make PDAC an immunologically cold tumor. To convert PDAC into an immunologically hot tumor, tumor microenvironment could be considered alongside the tumor cells. We discuss the complex network of microenvironment molecular and cellular composition and explore how they can be targeted to overcome immuno-therapeutic challenges.