Skin diseases, a significant category in the medical field, have always been challenging to diagnose and have a high misdiagnosis rate. Deep learning for skin disease classification has considerable value in clinical diagnosis and treatment. This study proposes a skin disease classification model based on multi-scale channel attention. The network architecture of the model consists of three main parts: an input module, four processing blocks, and an output module. Firstly, the model has improved the pyramid segmentation attention module to extract multi-scale features of the image entirely. Secondly, the reverse residual structure is used to replace the residual structure in the backbone network, and the attention module is integrated into the reverse residual structure to achieve better multi-scale feature extraction. Finally, the output module consists of an adaptive average pool and a fully connected layer, which convert the aggregated global features into several categories to generate the final output for the classification task. To verify the performance of the proposed model, this study used two commonly used skin disease datasets, ISIC2019 and HAM10000, for validation. The experimental results showed that the accuracy of this study was 77.6% on the ISIC2019 skin disease series dataset and 88.2% on the HAM10000 skin disease dataset. External validation data was added for evaluation to validate the model further, and the comprehensive evaluation results proved the effectiveness of the proposed model in this paper.