An imputation algorithm for human leukocyte antigen (HLA) is helpful for exploring novel disease associations. However, population-specific HLA imputation references are essential for achieving high imputation accuracy. In this study, a subset of 1012 individuals from the Taiwan Biobank (TWB) who underwent both whole-genome SNP array and NGS-based HLA typing were used to establish Taiwanese HLA imputation references. The HIBAG package was used to generate the imputation references for eight HLA loci at a two- and three-field resolution. Internal validation was carried out to evaluate the call threshold and accuracy for each HLA gene. HLA class II genes found to be associated with rheumatoid arthritis (RA) were validated in this study by the imputed HLA alleles. Our Taiwanese population-specific references achieved average HLA imputation accuracies of 98.11% for two-field and 98.08% for three-field resolution. The frequency distribution of imputed HLA alleles among 23,972 TWB subjects were comparable with PCR-based HLA alleles in general Taiwanese reported in the allele frequency net database. We replicated four common HLA alleles (HLA-DRB103:01, DRB104:05, DQA103:03, and DQB104:01) significantly associated with RA. The population-specific references provide an informative tool to investigate the associations of HLA variants and human diseases in large-scale population-based studies.