Individual recognition of Holstein cows is the basis for realizing precision dairy farming. Current machine vision individual recognition systems usually rely on fixed vertical illumination and top-view camera perspectives or require complex camera systems, and these requirements limit their promotion in practical applications. To solve this problem, a lightweight Holstein cow individual recognition feature extraction network named CowBackNet is designed in this paper. This network is not affected by camera angle and lighting changes and is suitable for farm environments. Secondly, a fusion multi-attention mechanism approach was adopted to integrate the attention mechanism, inverse residual structure, and depth-separable convolution technique to design a new feature extraction module, LightCBAM. This module was placed in the corresponding layer of CowBackNet to enhance the model's ability to extract the key features of the cow's back image from different viewpoints. In addition, the CowBack dataset was constructed in this study to verify the model's ability to be applied in real scenarios, containing Holstein cowback images in real production environments under different viewpoints. The experimental results show that when using CowBackNet as a feature extraction network, the recognition accuracy reaches 88.30%, FLOPs are 0.727 G, and the model size is only 6.096 MB. Compared with the classical EfficientNetV2, the accuracy of CowBackNet is improved by 11.69%, the FLOPs are reduced by 0.001 G, and the number of parameters is also reduced by 14.6%. Therefore, the model developed in this paper shows good robustness in shooting angle, light change, and real production data, which not only improves the recognition accuracy but also optimizes the computational efficiency of the model, which is of great practical application value for realizing precision farming.