Novel method for the recognition of Jinnan cattle action using bottleneck attention enhanced two-stream neural network
Keywords:
Jinnan cattle, action recognition, bottleneck attention, two-stream neural networkAbstract
Effective and accurate action recognition is essential to the intelligent breeding of the Jinnan cattle. However, there are still several challenges in the current Jinnan cattle action recognition. Traditional methods are based on manual characteristics and low recognition accuracy. This study is aimed at the efficient and accurate development of Jinnan cattle action recognition methods to overcome existing problems and support intelligent breeding. The acquired data from the previous methods contain a lot of noise, which will cause individual cattle to have excessive behaviors due to unsuitability. Concerning the high labor costs, low efficiency, and low model accuracy of the above approaches, this study developed a bottleneck attention-enhanced two-stream (BATS) Jinnan cattle action recognition method. It primarily comprises a Spatial Stream Subnetwork, a Temporal Stream Subnetwork, and a Bottleneck Attention Module. It can capture the spatial-channel dependencies in RGB and optical flow two branches respectively, so as to extract richer and more robust features. Finally, the decision of the two branches can be fused to gain improved cattle action recognition performance. Compared with the traditional methods, the model proposed in this study has achieved state-of-the-art recognition performance, and the accuracy of motion recognition was 96.53%, which was 4.60% higher than other models. This method significantly improves the efficiency and accuracy of behavior recognition and provides an important research foundation and direction for the development of higher-level behavior analysis models in the future development of smart animal husbandry. Keywords: Jinnan cattle, action recognition, bottleneck attention, two-stream neural network DOI: 10.25165/j.ijabe.20241703.8202 Citation: Hao W L, Han M, Zhang K, Zhang L, Hao W B, Li F Z, et al. Novel method for the recognition of Jinnan cattle action using bottleneck attention enhanced two-stream neural network. Int J Agric & Biol Eng, 2024; 17(3): 203-210.References
[1] Cairo F C, Pereira L G R, Campos M M, Tomich T R, Coelho S G, Lage C A, et al. Applying machine learning techniques on feeding behavior data for early estrus detection in dairy heifers. Computers and Electronics in Agriculture, 2020; 179: 105855.
[2] Tang J L, Yang G X, Sun Y R, Xin J, He D J. Salient object detection of dairy goats in farm image based on background and foreground priors. Neurocomputing, 2019; 332: 270–282.
[3] Shang C, Wu F, Wang M L, Gao Q. Cattle behavior recognition based on feature fusion under a dual attention mechanism. Journal of Visual Communication and Image Representation, 2022; 85: 103524.
[4] Heo E J, Ahn S J, Choi K S. Real-time cattle action recognition for estrus detection. KSII Transactions on Internet and Information Systems (TIIS), 2019; 13(4): 2148–2161.
[5] Nguyen C, Wang D, Von Richter K, Valencia P, Alvarenga F A P, Bishop-Hurley G. Video-based cattle identification and action recognition. In: 2021 Digital Image Computing: Techniques and Applications (DICTA), Gold Coast: IEEE, 2021; pp.1–5.
[6] Carreira J, Zisserman A. Quo vadis, action recognition? A new model and the kinetics dataset. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition, 2017; pp.4724–4733.
[7] Yao G L, Tao L, Zhong J D. A review of convolutional-neural-network-based action recognition. Pattern Recognition Letters, 2019; 118: 14–22.
[8] Zhang H B, Zhang Y X, Zhong B N, Lei Q, Yang L J, Du J X, et al. A comprehensive survey of vision-based human action recognition methods. Sensors, 2019; 19(5): 1005.
[9] Ahn S J, Ko D M, Heo E J, Choi K S. Real-time cow action recognition based on motion history image feature. In: 2018 IEEE International Conference on Consumer Electronics (ICCE), Las Vegas: IEEE, 2018; pp.1–2.
[10] Wu D H, Wu Q, Yin X Q, Jiang B, Wang H, He D J, et al. Lameness detection of dairy cows based on the YOLOv3 deep learning algorithm and a relative step size characteristic vector. Biosystems Engineering, 2020; 89: 150–163.
[11] Jiang B, Yin X Q, Song H B. Single-stream long-term optical flow convolution network for action recognition of lameness dairy cow. Computers and Electronics in Agriculture, 2020; 175: 105536.
[12] Shen W Z, Hu H Q, Dai B S, Wei X L, Sun J. Individual identification of dairy cows based on convolutional neural networks. Multimedia Tools and Applications, 2020; 79: 14711–14724.
[13] Tassinari P, Bovo M, Benni S, Franzoni S, Poggi M, Mammi L M E, et al. A computer vision approach based on deep learning for the detection of dairy cows in free stall barn. Computers and Electronics in Agriculture, 2021; 182: 106030.
[14] Zhang X D, Kang X, Feng N N, Liu G. Automatic recognition of dairy cow mastitis from thermal images by a deep learning detector. Computers and Electronics in Agriculture, 2020; 178: 105754.
[15] Hu H Q, Dai B S, Shen W Z, Wei X L, Sun J, Li R Z, et al. Cow identification based on fusion of deep parts features. Biosystems Engineering, 2020; 192: 245–256.
[16] Jiang B, Wu Q, Yin X Q, Wu D H, Song H B, He D J. FlyYOLOv3 deep learning for key parts of dairy cow body detection. Computers and Electronics in Agriculture, 2019; 166: 104982.
[17] Bezen R, Edan Y, Halachmi I. Computer vision system for measuring individual cow feed intake using RGB-D camera and deep learning algorithms. Computers and Electronics in Agriculture, 2020; 172: 105345.
[18] Achour B, Belkadi M, Filali I, Laghrouche M, Lahdir M. Image analysis for individual identification and feeding behaviour monitoring of dairy cows based on Convolutional Neural Networks (CNN). Biosystems Engineering, 2020; 198: 31–49.
[19] Wu D H, Wang Y F, Han M X, Song L, Shang Y Y, Zhang X Y, et al. Using a CNN-LSTM for basic behaviors detection of a single dairy cow in a complex environment. Computers and Electronics in Agriculture, 2021; 182: 106016.
[20] Simonyan K, Zisserman A. Two-stream convolutional networks for action recognition in videos. In: NIPS’14: Proceedings of the 27th International Conference on Neural Information Processing Systems, 2014; 1: 568–576.
[21] Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks. Comuniacations of the ACM, 2017; 60(6): 84–90.
[22] Park J, Woo S, Lee J-Y, Kweon I, BAM: Bottleneck attention module. arXiv in Press, 2018. arXiv: 1807.06514.
[23] Sandler M, Howard A, Zhu M L, Zhmoginov A, Chen L C. MobileNetV2: Inverted residuals and linear bottlenecks. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City: IEEE, 2018; pp.4510–4520.
[24] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv Preprint, 2014. arXiv: 1409.1556.
[25] Iandola F N, Han S, Moskewicz M W, Ashraf K, Dally W J, Keutzer K. SqueezeNet: Alexnet-level accuracy with 50x fewer parameters and <0.5 MB model size. arXiv Preprint, 2016. arXiv: 1602.07360.
[26] He K M, Zhang X Y, Ren S Q, Sun J. Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016; pp.770–778.
[2] Tang J L, Yang G X, Sun Y R, Xin J, He D J. Salient object detection of dairy goats in farm image based on background and foreground priors. Neurocomputing, 2019; 332: 270–282.
[3] Shang C, Wu F, Wang M L, Gao Q. Cattle behavior recognition based on feature fusion under a dual attention mechanism. Journal of Visual Communication and Image Representation, 2022; 85: 103524.
[4] Heo E J, Ahn S J, Choi K S. Real-time cattle action recognition for estrus detection. KSII Transactions on Internet and Information Systems (TIIS), 2019; 13(4): 2148–2161.
[5] Nguyen C, Wang D, Von Richter K, Valencia P, Alvarenga F A P, Bishop-Hurley G. Video-based cattle identification and action recognition. In: 2021 Digital Image Computing: Techniques and Applications (DICTA), Gold Coast: IEEE, 2021; pp.1–5.
[6] Carreira J, Zisserman A. Quo vadis, action recognition? A new model and the kinetics dataset. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition, 2017; pp.4724–4733.
[7] Yao G L, Tao L, Zhong J D. A review of convolutional-neural-network-based action recognition. Pattern Recognition Letters, 2019; 118: 14–22.
[8] Zhang H B, Zhang Y X, Zhong B N, Lei Q, Yang L J, Du J X, et al. A comprehensive survey of vision-based human action recognition methods. Sensors, 2019; 19(5): 1005.
[9] Ahn S J, Ko D M, Heo E J, Choi K S. Real-time cow action recognition based on motion history image feature. In: 2018 IEEE International Conference on Consumer Electronics (ICCE), Las Vegas: IEEE, 2018; pp.1–2.
[10] Wu D H, Wu Q, Yin X Q, Jiang B, Wang H, He D J, et al. Lameness detection of dairy cows based on the YOLOv3 deep learning algorithm and a relative step size characteristic vector. Biosystems Engineering, 2020; 89: 150–163.
[11] Jiang B, Yin X Q, Song H B. Single-stream long-term optical flow convolution network for action recognition of lameness dairy cow. Computers and Electronics in Agriculture, 2020; 175: 105536.
[12] Shen W Z, Hu H Q, Dai B S, Wei X L, Sun J. Individual identification of dairy cows based on convolutional neural networks. Multimedia Tools and Applications, 2020; 79: 14711–14724.
[13] Tassinari P, Bovo M, Benni S, Franzoni S, Poggi M, Mammi L M E, et al. A computer vision approach based on deep learning for the detection of dairy cows in free stall barn. Computers and Electronics in Agriculture, 2021; 182: 106030.
[14] Zhang X D, Kang X, Feng N N, Liu G. Automatic recognition of dairy cow mastitis from thermal images by a deep learning detector. Computers and Electronics in Agriculture, 2020; 178: 105754.
[15] Hu H Q, Dai B S, Shen W Z, Wei X L, Sun J, Li R Z, et al. Cow identification based on fusion of deep parts features. Biosystems Engineering, 2020; 192: 245–256.
[16] Jiang B, Wu Q, Yin X Q, Wu D H, Song H B, He D J. FlyYOLOv3 deep learning for key parts of dairy cow body detection. Computers and Electronics in Agriculture, 2019; 166: 104982.
[17] Bezen R, Edan Y, Halachmi I. Computer vision system for measuring individual cow feed intake using RGB-D camera and deep learning algorithms. Computers and Electronics in Agriculture, 2020; 172: 105345.
[18] Achour B, Belkadi M, Filali I, Laghrouche M, Lahdir M. Image analysis for individual identification and feeding behaviour monitoring of dairy cows based on Convolutional Neural Networks (CNN). Biosystems Engineering, 2020; 198: 31–49.
[19] Wu D H, Wang Y F, Han M X, Song L, Shang Y Y, Zhang X Y, et al. Using a CNN-LSTM for basic behaviors detection of a single dairy cow in a complex environment. Computers and Electronics in Agriculture, 2021; 182: 106016.
[20] Simonyan K, Zisserman A. Two-stream convolutional networks for action recognition in videos. In: NIPS’14: Proceedings of the 27th International Conference on Neural Information Processing Systems, 2014; 1: 568–576.
[21] Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks. Comuniacations of the ACM, 2017; 60(6): 84–90.
[22] Park J, Woo S, Lee J-Y, Kweon I, BAM: Bottleneck attention module. arXiv in Press, 2018. arXiv: 1807.06514.
[23] Sandler M, Howard A, Zhu M L, Zhmoginov A, Chen L C. MobileNetV2: Inverted residuals and linear bottlenecks. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City: IEEE, 2018; pp.4510–4520.
[24] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv Preprint, 2014. arXiv: 1409.1556.
[25] Iandola F N, Han S, Moskewicz M W, Ashraf K, Dally W J, Keutzer K. SqueezeNet: Alexnet-level accuracy with 50x fewer parameters and <0.5 MB model size. arXiv Preprint, 2016. arXiv: 1602.07360.
[26] He K M, Zhang X Y, Ren S Q, Sun J. Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016; pp.770–778.
Downloads
Published
2024-07-11
How to Cite
Hao, W., Han, M., Zhang, K., Zhang, L., Hao, W., Li, F., & Liu, Z. (2024). Novel method for the recognition of Jinnan cattle action using bottleneck attention enhanced two-stream neural network. International Journal of Agricultural and Biological Engineering, 17(3), 203–210. Retrieved from https://ijabe.migration.pkpps03.publicknowledgeproject.org/index.php/ijabe/article/view/8202
Issue
Section
Information Technology, Sensors and Control Systems
License
IJABE is an international peer reviewed open access journal, adopting Creative Commons Copyright Notices as follows.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
