Context-aware citation recommendation aims to automatically predict suitable citations for a given citation context, which is essentially helpful for researchers when writing scientific papers. In existing neural network-based approaches, overcorrelation in the weight matrix influences semantic similarity, which is a difficult problem to solve. In this paper, we propose a novel context-aware citation recommendation approach that can essentially improve the orthogonality of the weight matrix and explore more accurate citation patterns. We quantitatively show that the various reference patterns in the paper have interactional features that can significantly affect link prediction. We conduct experiments on the CiteSeer datasets. The results show that our model is superior to baseline models in all metrics.