Pancreaticobiliary tract cancer has a poor prognosis with unmet needs in a new target treatment. Some studies have reported that an enhancement of T-cell immunity is associated with a good prognosis. The aim of this study is to investigate the immunoprofile as a prognostic marker of pancreaticobiliary tract cancers. Unresectable pancreatic ductal adenocarcinoma (PDAC, n = 80) and biliary tract cancer (BTC, n = 74) diagnosed between January 2012 and December 2018 in Samsung Medical Center were analyzed. Expression levels of CD8, FOXP3, PD-1, PD-L1, and CXCL13 in PDAC and BTC tissue samples were examined with immunohistochemical staining, which was evaluated with various clinical factors. In PDAC, higher degree of PD-L1 expression was significantly associated with shorter overall survival (OS) (p = 0.0095). On the other hand, higher infiltrations of PD-1 immune cells (p = 0.0002) and CD8 T cells (p = 0.0067) were associated with longer OS. In BTC, higher FOXP3 (p = 0.0343) and CD8 (p = 0.0028) cell infiltrations were associated with better survival. Low infiltration of CD8 (p = 0.0148), FOXP3 (p = 0.0208), PD-1 (p = 0.0318) cells in PDAC, and FOXP3 cells (p = 0.005) in BTC were considerably related to metastasis. In a combined evaluation of clinical factors and immunoprofiles, univariate analysis revealed that operation after chemotherapy (p < 0.0001), mass size (p = 0.0004), metastasis (p = 0.006), PD-L1 (p < 0.0001), PD-1 (p = 0.003) and CD8 (p = 0.0063) was significantly associated with OS in PDAC, and CD8 (p = 0.007) was statistically related to OS in BTC. In multivariate analysis, prognostic factors were operation after chemotherapy (p = 0.021) in PDAC and CD8 (p = 0.037) in BTC. Therefore, immunoprofile analysis of cells expressing CD8, FOXP3, PD-1, and PD-L1 might have prognostic values in patients with pancreaticobiliary tract cancers.