Biliary tumors showing intraductal papillary growth (Pap-BTs) include intraductal papillary neoplasm of the bile duct (IPNB) and papillary cholangiocarcinoma (CC). A differential diagnosis between IPNB and papillary CC currently remains challenging. The aim of the present study is to identify histological features and immunohistochemical markers of malignant potential such as tumor invasion in Pap-BTs. Subjects comprised 37 patients with Pap-BT (intrahepatic and perihilar [proximal], 27: 17 noninvasive and 10 invasive; distal, 10: all invasive). We examined histological features and the expression of p53, enhancer of zeste homolog 2, insulin-like growth factor II mRNA-binding protein 3 (IMP3), and DNA methyltransferase-1 in the intraductal area in Pap-BTs. Noninvasive Pap-BT was characterized by the presence of a low-grade dysplastic area, edematous stroma, and the absence of necrosis. The expression of p53, enhancer of zeste homolog 2, IMP3, and DNA methyltransferase-1 was significantly weaker in noninvasive Pap-BTs than in invasive Pap-BTs (P<.01). Diffuse cytoplasmic IMP3 expression was absent in noninvasive Pap-BTs. IMP3 showed the greatest specificity to predict a presence of invasion. A heatmap demonstrated that proximal noninvasive Pap-BTs and distal Pap-BTs may be completely different. In bile duct biopsies, the expression of IMP3 was the most precise predictor of invasion in Pap-BTs. In conclusion, Pap-BTs may be separated into 3 subgroups: (1) proximal noninvasive Pap-BT, corresponding to IPNB; (2) distal invasive Pap-BT, corresponding to papillary CC; and (3) the remaining Pap-BT including IPNB with associated adenocarcinomas, based on histological and immunohistochemical features. IMP3 may be a useful marker for predicting invasion in Pap-BT.