Skin is the outer cover of our body, which protects vital organs from harm. This important body part is often affected by a series of infections caused by fungus, bacteria, viruses, allergies, and dust. Millions of people suffer from skin diseases. It is one of the common causes of infection in sub-Saharan Africa. Skin disease can also be the cause of stigma and discrimination. Early and accurate diagnosis of skin disease can be vital for effective treatment. Laser and photonics-based technologies are used for the diagnosis of skin disease. These technologies are expensive and not affordable, especially for resource-limited countries like Ethiopia. Hence, image-based methods can be effective in reducing cost and time. There are previous studies on image-based diagnosis for skin disease. However, there are few scientific studies on tinea pedis and tinea corporis. In this study, the convolution neural network (CNN) has been used to classify fungal skin disease. The classification was carried out on the four most common fungal skin diseases: tinea pedis, tinea capitis, tinea corporis, and tinea unguium. The dataset consisted of a total of 407 fungal skin lesions collected from Dr. Gerbi Medium Clinic, Jimma, Ethiopia. Normalization of image size, conversion of RGB to grayscale, and balancing the intensity of the image have been carried out. Images were normalized to three sizes: 120 × 120, 150 × 150, and 224 × 224. Then, augmentation was applied. The developed model classified the four common fungal skin diseases with 93.3% accuracy. Comparisons were made with similar CNN architectures: MobileNetV2 and ResNet 50, and the proposed model was superior to both. This study may be an important addition to the very limited work on the detection of fungal skin disease. It can be used to build an automated image-based screening system for dermatology at an initial stage.