Salivary gland neoplasms (SGNs) represent a group of human neoplasms characterized by a remarkable cytomorphologic diversity, which frequently poses diagnostic challenges. Accurate histologic categorization of salivary gland tumors is crucial to make precise diagnoses and guide decisions regarding patient management. Within the scope of this study, a computer-aided diagnosis model using Vision Transformer (ViT), a cutting-edge deep learning model in computer vision, was developed to accurately classify the most prevalent subtypes of SGNs. These subtypes include pleomorphic adenoma, myoepithelioma, Warthin tumor, basal cell adenoma, oncocytic adenoma, cystadenoma, mucoepidermoid carcinoma, and salivary adenoid cystic carcinoma. The data set comprised 3046 whole slide images of histologically confirmed salivary gland tumors, encompassing nine distinct tissue categories. SGN-ViT exhibited impressive performance in classifying the eight salivary gland tumors, achieving an accuracy of 0.9966, an area under the receiver operating characteristic curve value of 0.9899, precision of 0.9848, recall of 0.9848, and an F1 score of 0.9848. Diagnostic performance of SGN-ViT surpassed that of benchmark models. In a subset of 100 whole slide images, SGN-ViT demonstrated comparable diagnostic performance to that of the chief pathologist while significantly reducing the diagnosis time. These observations indicate that SGN-ViT holds the potential to serve as a valuable computer-aided diagnostic tool for salivary gland tumors, enhancing the diagnostic accuracy of junior pathologists.