The most dangerous form of cancer is breast cancer. This disease is life-threatening because of its aggressive nature and high death rates. Therefore, early discovery increases the patient's survival. Mammography has recently been recommended as diagnosis technique. Mammography, is expensive and exposure the person to radioactivity. Thermography is a less invasive and affordable technique that is becoming increasingly popular. Considering this, a recent deep learning-based breast cancer diagnosis approach is executed by thermography images. Initially, thermography images are chosen from online sources. The collected thermography images are being preprocessed by Contrast Limited Adaptive Histogram Equalization (CLAHE) and contrasting enhancement methods to improve the quality and brightness of the images. Then, the optimal binary thresholding is done to segment the preprocessed images, where optimized the thresholding value using developed Rock Hyraxes Dandelion Algorithm Optimization (RHDAO). A newly implemented deep learning structure StackVRDNet is used for further processing breast cancer diagnosing using thermography images. The segmented images are fed to the StackVRDNet framework, where the Visual Geometry Group (VGG16), Resnet, and DenseNet are employed for constructing this model. The relevant features are extracted usingVGG16, Resnet, and DenseNet, and then obtain stacked weighted feature pool from the extracted features, where the weight optimization is done with the help of RHDAO. The final classification is performed using StackVRDNet, and the diagnosis results are obtained at the final layer of VGG16, Resnet, and DenseNet. A higher scoring method is rated for ensuring final diagnosis results. Here, the parameters present within the VGG16, Resnet, and DenseNet are optimized via the RHDAO to improve the diagnosis results. The simulation outcomes of the developed model achieve 97.05% and 86.86% in terms of accuracy and precision, respectively. The effectiveness of the designed methd is being analyzed via the conventional breast cancer diagnosis models in terms of various performance measures.