Pneumonia remains a significant global health challenge, necessitating timely and accurate diagnosis for effective treatment. In recent years, deep learning techniques have emerged as powerful tools for automating pneumonia detection from chest X-ray images. This paper provides a comprehensive investigation into the application of deep learning for pneumonia detection, with an emphasis on overcoming the challenges posed by imbalanced datasets. The study evaluates the performance of various deep learning architectures, including visual geometry group (VGG), residual networks (ResNet), and Vision Transformers (ViT) along with strategies to mitigate the impact of imbalanced dataset, on publicly available datasets such as the Chest X-Ray Images (Pneumonia) dataset, BRAX dataset, and CheXpert dataset. Additionally, transfer learning from pre-trained models, such as ImageNet, is investigated to leverage prior knowledge for improved performance on pneumonia detection tasks. Our investigation extends to zero-shot and few-shot learning experiments on different geographical regions. The study also explores semi-supervised learning methods, including the Mean Teacher algorithm, to utilize unlabeled data effectively. Experimental results demonstrate the efficacy of transfer learning, data augmentation, and balanced weight in addressing imbalanced datasets, leading to improved accuracy and performance in pneumonia detection. Our findings emphasize the importance of selecting appropriate strategies based on dataset characteristics, with semi-supervised learning showing particular promise in leveraging unlabeled data. The findings highlight the potential of deep learning techniques in revolutionizing pneumonia diagnosis and treatment, paving the way for more efficient and accurate clinical workflows in the future.