Electrocardiogram (ECG) signals contain complex and diverse features, serving as a crucial basis for arrhythmia diagnosis. The subtle differences in characteristics among various types of arrhythmias, coupled with class imbalance issues in datasets, often hinder existing models from effectively capturing key information within these complex signals, leading to a bias towards normal classes. To address these challenges, this paper proposes a method for arrhythmia classification based on a multi-branch, multi-head attention temporal convolutional network (MB-MHA-TCN). The model integrates three convolutional branch layers with different kernel sizes and dilation rates to capture features across varying temporal scales. A multi-head self-attention mechanism dynamically allocates weights, integrating features and correlations from different branches to enhance the recognition capability for difficult-to-classify samples. Additionally, the temporal convolutional network employs multi-layer dilated convolutions to progressively expand the receptive field for extracting long-term dependencies. To tackle data imbalance, a novel data augmentation strategy is implemented, and focal loss is utilized to increase the weight of minority classes, while Bayesian optimization is employed to fine-tune the model's hyperparameters. The results from five-fold cross-validation on the MIT-BIH Arrhythmia Database demonstrate that the proposed method achieves an overall accuracy of 98.75%, precision of 96.60%, sensitivity of 97.21%, and F1 score of 96.89% across five categories of ECG signals. Compared to other studies, this method exhibits superior performance in arrhythmia classification, significantly improving the recognition rate of minority classes.