PROBLEM: The most prevalent cancer in women is breast cancer (BC), and effective treatment depends on being detected early. Many people seek medical imaging techniques to help in the early detection of problems, but results often need to be corrected for increased accuracy. AIM: A new deep learning approach for medical images is applied in the detection of BC in this paper. Early detection is carried out through the proposed method using a combination of Convolutional Neural Network (CNNs) with feature selection and fusion methods. METHODS: The proposed method may decrease the mortality rate due to the early-stage detection of BC with high precision. In this work, the proposed Deep Learning Framework (DLF) uses many levels of artificial neural networks to sort images of BC into categories correctly. RESULTS: This proposed method further increases the scalability of convolutional recurrent networks. It also achieved 94.93 % accuracy, 93.66 % precision, 89.21 % recall and 98.86 % F1-score. Through this approach, cancer tumors in a specific location can be detected more accurately. CONCLUSION: The existing methods are dependent mainly on manually selecting and extracting features. The proposed framework automatically learns and finds relevant features from images that result in outperforming existing methods.