BACKGROUND: Determining the status of breast cancer susceptibility genes () is crucial for guiding breast cancer treatment. Nevertheless, the need for genetic testing among breast cancer patients remains unmet due to high costs and limited resources. This study aimed to develop a Bi-directional Self-Attention Multiple Instance Learning (BiAMIL) algorithm to detect status from hematoxylin and eosin (H&E) pathological images. METHODS: A total of 319 histopathological slides from 254 breast cancer patients were included, comprising two dependent cohorts. Following image pre-processing, 633,484 tumor tiles from the training dataset were employed to train the self-developed deep-learning model. The performance of the network was evaluated in the internal and external test sets. RESULTS: BiAMIL achieved AUC values of 0.819 (95% CI [0.673-0.965]) in the internal test set, and 0.817 (95% CI [0.712-0.923]) in the external test set. To explore the relationship between status and interpretable morphological features in pathological images, we utilized Class Activation Mapping (CAM) technique and cluster analysis to investigate the connections between gene mutation status and tissue and cell features. Significantly, we observed that tumor-infiltrating lymphocytes and the morphological characteristics of tumor cells appeared to be potential features associated with status. CONCLUSIONS: An interpretable deep neural network model based on the attention mechanism was developed to predict the status in breast cancer. Keywords: Breast cancer, , deep learning, self-attention, interpretability.