Parallel channel and position attention-guided feature pyramid for pig face posture detection
Keywords:
objection detection, attention mechanism, feature pyramid network, face posture detection, pigAbstract
The area of the pig's face contains rich biological information, such as eyes, nose, and ear. The high-precision detection of pig face postures is crucial to the identification of pigs, and it can also provide fundamental archival information for the study of abnormal behavioral characteristics and regularities. In this study, a series of attention blocks were embedded in Feature Pyramid Network (FPN) for automatic detection of the pig face posture in group-breeding environments. Firstly, the Channel Attention Block (CAB) and Position Attention Block (PAB) were proposed to capture the channel dependencies and the pixel-level long-range relationships, respectively. Secondly, a variety of attention modules are proposed to effectively combine the two kinds of attention information, specifically including Parallel Channel Position (PCP), Cascade Position Channel (CPC), and Cascade Channel Position (CCP), which fuse the channel and position attention information in both parallel or cascade ways. Finally, the verification experiments on three task networks with two backbone networks were conducted for different attention blocks or modules. A total of 45 pigs in 8 pigpens were used as the research objects. Experimental results show that attention-based models perform better. Especially, with Faster Region Convolutional Neural Network (Faster R-CNN) as the task network and ResNet101 as the backbone network, after the introduction of the PCP module, the Average Precision (AP) indicators of the face poses of Downward with head-on face (D-O), Downward with lateral face (D-L), Level with head-on face (L-O), Level with lateral face (L-L), Upward with head-on face (U-O), and Upward with lateral face (U-L) achieve 91.55%, 90.36%, 90.10%, 90.05%, 85.96%, and 87.92%, respectively. Ablation experiments show that the PAB attention block is not as effective as the CAB attention block, and the parallel combination method is better than the cascade manner. Taking Faster R-CNN as the task network and ResNet101 as the backbone network, the heatmap visualization of different layers of FPN before and after adding PCP shows that, compared with the non-PCP module, the PCP module can more easily aggregate denser and richer contextual information, this, in turn, enhances long-range dependencies to improve feature representation. At the same time, the model based on PCP attention can effectively detect the pig face posture of different ages, different scenes, and different light intensities, which can help lay the foundation for subsequent individual identification and behavior analysis of pigs. Keywords: objection detection, attention mechanism, feature pyramid network, face posture detection, pig DOI: 10.25165/j.ijabe.20221506.7329 Citation: Hu Z W, Yan H W, Lou T T. Parallel channel and position attention-guided feature pyramid for pig face posture detection. Int J Agric & Biol Eng, 2022; 15(6): 222–234.References
[1] Hansen M F, Smith M L, Smith L N, Salter M G, Baxte E M, Farish M, et al. Towards on-farm pig face recognition using convolutional neural networks. Computers in Industry, 2018; 98: 145–152.
[2] Marsot M, Mei J Q, Shan X C, Ye L Y, Feng P, Yan X J, et al. An adaptive pig face recognition approach using Convolutional Neural Networks. Computers and Electronics in Agriculture,2020; 173: 105386. doi: 10.1016/j.compag.2020.105386.
[3] Zhang K F, Li D, Huang J Y, Chen Y F. Automated video behavior recognition of pigs using two-stream convolutional networks. Sensors, 2020; 20(4): 1085. doi: 10.3390/s20041085.
[4] Condotta, I C, Brown-Brandl T M, Silva-Miranda K O, Stinn J P. Evaluation of a depth sensor for mass estimation of growing and finishing pigs. Biosystems Engineering, 2018; 173: 11–18.
[5] Valletta J J, Torney C, Kings M, Thornton A, Madde J. Applications of machine learning in animal behavior studies. Animal Behavior, 2017; 124: 203–220.
[6] Nasirahmadi A, Sturm B, Olsson A C, Jeppsson K H, Müller S, Edwards S, et al. Automatic scoring of lateral and sternal lying posture in grouped pigs using image processing and Support Vector Machine. Computers and electronics in agriculture, 2019; 156: 475–481.
[7] Shi C, Zhang J L, Teng G H. Mobile measuring system based on LabVIEW for pig body components estimation in a large-scale farm. Computers and electronics in agriculture, 2019; 156: 399–405.
[8] Chen C, Zhu W X, Liu D, Steibel J, Siegford J, Wurtz K, et al. Detection of aggressive behaviors in pigs using a RealSence depth sensor. Computers and Electronics in Agriculture, 2019; 166, 105003. doi: 10.1016/j.compag.2019.105003.
[9] da Fonseca F N, Abe J M, de Alencar Nääs I, da Silva Cordeiro A F, do Amaral F V, Ungaro H C. Automatic prediction of stress in piglets (Sus Scrofa) using infrared skin temperature. Computers and Electronics in Agriculture, 2020; 168: 105148. doi: 10.1016/j.compag.2019.105148.
[10] Sun L Q, Chen S H, Liu T, Liu C H, Liu Y. Pig target tracking algorithm based on multi-channel color feature fusion. Int J Agric & Biol Eng, 2020; 13(3): 180–185.
[11] Wu X W, Sahoo D, Hoi S C H. Recent advances in deep learning for object detection. Neurocomputing, 2020; 396: 39–64.
[12] Zhang Y Q, Chu J, Leng L, Miao J. Mask-refined R-CNN: A network for refining object details in instance segmentation. Sensors, 2020; 20(4), 1010. doi: 10.3390/s20041010.
[13] Kamilaris A, Prenafeta-Boldú F X. Deep learning in agriculture: A survey. Computers and Electronics in Agriculture, 2018; 147: 70–90.
[14] Tassinari P, Bovo M, Benni S, Franzoni S, Poggi M, Mammi L M E, et al. A computer vision approach based on deep learning for the detection of dairy cows in free stall barn. Computers and Electronics in Agriculture, 2021; 182: 106030. doi: 10.1016/j.compag.2021.106030.
[15] Jiang M, Rao Y, Zhang J Y, Shen Y M. Automatic behavior recognition of group-housed goats using deep learning. Computers and Electronics in Agriculture, 2020; 177: 105706. doi: 10.1016/j.compag.2020.105706.
[16] Su D, Qiao Y, Kong H, Sukkarieh S. Real time detection of inter-row ryegrass in wheat farms using deep learning. Biosystems Engineering, 2021; 204: 198–211.
[17] Chen C, Zhu W, Steibel J, Siegford J, Wurtz K, Han J, et al. Recognition of aggressive episodes of pigs based on convolutional neural network and long short-term memory. Computers and Electronics in Agriculture, 2020; 169: 105166. doi: 10.1016/j.compag.2019.105166.
[18] Chen C, Zhu W X, Steibel J P, Siegford J M, Han J, Norton T J. Classification of drinking and drinker-playing in pigs by a video-based deep learning method. Biosystems Engineering, 2020; 196: 1–14.
[19] Yang A, Huang H, Yang X, Li S, Chen C, Gan H, et al. Automated video analysis of sow nursing behavior based on fully convolutional network and oriented optical flow. Computers and Electronics in Agriculture, 2019; 167: 105048. doi: 10.1016/j.compag.2019.105048.
[20] Hu Z W, Yang H, Lou T T, Hu G, Xie Q Q, Huang J M. Extraction of pig contour based on fully convolutional networks. Journal of South China Agricultural University 2018; 39(6): 111–119. (in Chinese)
[21] Hu Z W, Yang H, Lou T T. Dual attention-guided feature pyramid network for instance segmentation of group pigs. Computers and Electronics in Agriculture, 2021; 186: 106140. doi: 10.1016/ j.compag.2021.106140.
[22] Tian M X, Guo H, Chen H, Wang Q, Long C J, Ma Y H. Automated pig counting using deep learning. Computers and Electronics in Agriculture, 2019; 163 104840. doi: 10.1016/j.compag.2019.05.049.
[23] Nasirahmadi A, Sturm B, Edwards S, Jeppsson K H, Olsson A C, Müller S, et al. Deep learning and machine vision approaches for posture detection of individual pigs. Sensors, 2019; 19(17): 3738. doi: 10.3390/s19173738.
[24] Zheng C, Zhu X, Yang X, Wang L, Tu S, Xue Y. Automatic recognition of lactating sow postures from depth images by deep learning detector. Computers and Electronics in Agriculture, 2018; 147: 51–63.
[25] Tong W, Chen W T, Han W, Li X J, Wang L Z. Channel-attention-based DenseNet network for remote sensing image scene classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2020; 13: 4121–4132.
[26] Huang G, Liu Z, Van Der Maaten L, Weinberger K Q. Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2017; pp.4700–4708.
[27] Chen C F, Gong D H, Wang H, Li Z F, Wong K Y K. Learning spatial attention for face super-resolution. IEEE Transactions on Image Processing, 2020; 30: 1219–1231.
[28] Woo S, Park J, Lee J Y, Kweon I S. Cbam: Convolutional block attention module. In Proceedings of the European conference on computer vision (ECCV), IEEE, 2018; pp.3–19.
[29] Park J, Woo S, Lee J Y, Kweon I S. Bam: Bottleneck attention module. arXiv, 2018; arXiv preprint arXiv:1807.06514.
[30] Roy A G, Navab N, Wachinger C. Concurrent spatial and channel ‘squeeze & excitation’ in fully convolutional networks. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, 2018; pp.421–429.
[31] Fu J, Liu J, Tian H J, Li Y, Bao Y J, Fang Z W, et al. Dual attention network for scene segmentation. In: Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach: IEEE, 2019; pp.3146–3154. doi: 10.1109/CVPR.2019.00326.
[32] Girshick R, Donahue J, Darrell T, Malik J. Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2014; pp.580–587.
[33] Girshick R. Fast r-cnn. In: Proceedings of the IEEE international conference on computer vision, 2015; pp.1440–1448.
[34] Ren S, He K, Girshick R, Sun J. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016; 39(6): 1137–1149.
[35] Lin T, Dollar P, Girshick R, He K, Hariharan B, Belongie S. Feature pyramid networks for object detection. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, 2019; pp.936–944.
[36] Rezatofighi H, Tsoi N, Gwak J, Sadeghian A, Reid I, Savarese S. Generalized intersection over union: A metric and a loss for bounding box regression. In: Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, 2019; pp.658–666.
[37] Cai Z, Vasconcelos N. Cascade r-cnn: Delving into high quality object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2018; pp.6154–6162.
[38] Pang J M, Chen K, Shi J P, Feng H J, Ouyang W L, Lin D H. Libra R-CNN: Towards balanced learning for object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, 2019; pp.821–830.
[39] He K M, Zhang X Y, Ren S Q, Sun J. Deep Residual Learning for Image Recognition. computer vision and pattern recognition. In: 2016 IEEE Conference on Computer Vision and Patter Recognition (CVPR), IEEE, 2016; pp.770–778.
[40] Wu T Y, Tang S, Zhang R, Cao J, Zhang Y D. Cgnet: A light-weight context guided network for semantic segmentation. IEEE Transactions on Image Processing, 2020; 30: 1169–1179.
[41] Hu J, Shen L, Sun G. Squeeze-and-excitation networks. In: Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2018; 7132–7141.
[42] open-mmlab/mmdetection. Available: https://github.com/open-mmlab/ mmdetection. Accessed on [2022-01-15].
[43] Visual Object Classes Challenge 2012 (VOC 2012). Available: http://host.robots.ox.ac.uk/pascal/VOC/voc2012. Accessed on [2022-01-15].
[2] Marsot M, Mei J Q, Shan X C, Ye L Y, Feng P, Yan X J, et al. An adaptive pig face recognition approach using Convolutional Neural Networks. Computers and Electronics in Agriculture,2020; 173: 105386. doi: 10.1016/j.compag.2020.105386.
[3] Zhang K F, Li D, Huang J Y, Chen Y F. Automated video behavior recognition of pigs using two-stream convolutional networks. Sensors, 2020; 20(4): 1085. doi: 10.3390/s20041085.
[4] Condotta, I C, Brown-Brandl T M, Silva-Miranda K O, Stinn J P. Evaluation of a depth sensor for mass estimation of growing and finishing pigs. Biosystems Engineering, 2018; 173: 11–18.
[5] Valletta J J, Torney C, Kings M, Thornton A, Madde J. Applications of machine learning in animal behavior studies. Animal Behavior, 2017; 124: 203–220.
[6] Nasirahmadi A, Sturm B, Olsson A C, Jeppsson K H, Müller S, Edwards S, et al. Automatic scoring of lateral and sternal lying posture in grouped pigs using image processing and Support Vector Machine. Computers and electronics in agriculture, 2019; 156: 475–481.
[7] Shi C, Zhang J L, Teng G H. Mobile measuring system based on LabVIEW for pig body components estimation in a large-scale farm. Computers and electronics in agriculture, 2019; 156: 399–405.
[8] Chen C, Zhu W X, Liu D, Steibel J, Siegford J, Wurtz K, et al. Detection of aggressive behaviors in pigs using a RealSence depth sensor. Computers and Electronics in Agriculture, 2019; 166, 105003. doi: 10.1016/j.compag.2019.105003.
[9] da Fonseca F N, Abe J M, de Alencar Nääs I, da Silva Cordeiro A F, do Amaral F V, Ungaro H C. Automatic prediction of stress in piglets (Sus Scrofa) using infrared skin temperature. Computers and Electronics in Agriculture, 2020; 168: 105148. doi: 10.1016/j.compag.2019.105148.
[10] Sun L Q, Chen S H, Liu T, Liu C H, Liu Y. Pig target tracking algorithm based on multi-channel color feature fusion. Int J Agric & Biol Eng, 2020; 13(3): 180–185.
[11] Wu X W, Sahoo D, Hoi S C H. Recent advances in deep learning for object detection. Neurocomputing, 2020; 396: 39–64.
[12] Zhang Y Q, Chu J, Leng L, Miao J. Mask-refined R-CNN: A network for refining object details in instance segmentation. Sensors, 2020; 20(4), 1010. doi: 10.3390/s20041010.
[13] Kamilaris A, Prenafeta-Boldú F X. Deep learning in agriculture: A survey. Computers and Electronics in Agriculture, 2018; 147: 70–90.
[14] Tassinari P, Bovo M, Benni S, Franzoni S, Poggi M, Mammi L M E, et al. A computer vision approach based on deep learning for the detection of dairy cows in free stall barn. Computers and Electronics in Agriculture, 2021; 182: 106030. doi: 10.1016/j.compag.2021.106030.
[15] Jiang M, Rao Y, Zhang J Y, Shen Y M. Automatic behavior recognition of group-housed goats using deep learning. Computers and Electronics in Agriculture, 2020; 177: 105706. doi: 10.1016/j.compag.2020.105706.
[16] Su D, Qiao Y, Kong H, Sukkarieh S. Real time detection of inter-row ryegrass in wheat farms using deep learning. Biosystems Engineering, 2021; 204: 198–211.
[17] Chen C, Zhu W, Steibel J, Siegford J, Wurtz K, Han J, et al. Recognition of aggressive episodes of pigs based on convolutional neural network and long short-term memory. Computers and Electronics in Agriculture, 2020; 169: 105166. doi: 10.1016/j.compag.2019.105166.
[18] Chen C, Zhu W X, Steibel J P, Siegford J M, Han J, Norton T J. Classification of drinking and drinker-playing in pigs by a video-based deep learning method. Biosystems Engineering, 2020; 196: 1–14.
[19] Yang A, Huang H, Yang X, Li S, Chen C, Gan H, et al. Automated video analysis of sow nursing behavior based on fully convolutional network and oriented optical flow. Computers and Electronics in Agriculture, 2019; 167: 105048. doi: 10.1016/j.compag.2019.105048.
[20] Hu Z W, Yang H, Lou T T, Hu G, Xie Q Q, Huang J M. Extraction of pig contour based on fully convolutional networks. Journal of South China Agricultural University 2018; 39(6): 111–119. (in Chinese)
[21] Hu Z W, Yang H, Lou T T. Dual attention-guided feature pyramid network for instance segmentation of group pigs. Computers and Electronics in Agriculture, 2021; 186: 106140. doi: 10.1016/ j.compag.2021.106140.
[22] Tian M X, Guo H, Chen H, Wang Q, Long C J, Ma Y H. Automated pig counting using deep learning. Computers and Electronics in Agriculture, 2019; 163 104840. doi: 10.1016/j.compag.2019.05.049.
[23] Nasirahmadi A, Sturm B, Edwards S, Jeppsson K H, Olsson A C, Müller S, et al. Deep learning and machine vision approaches for posture detection of individual pigs. Sensors, 2019; 19(17): 3738. doi: 10.3390/s19173738.
[24] Zheng C, Zhu X, Yang X, Wang L, Tu S, Xue Y. Automatic recognition of lactating sow postures from depth images by deep learning detector. Computers and Electronics in Agriculture, 2018; 147: 51–63.
[25] Tong W, Chen W T, Han W, Li X J, Wang L Z. Channel-attention-based DenseNet network for remote sensing image scene classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2020; 13: 4121–4132.
[26] Huang G, Liu Z, Van Der Maaten L, Weinberger K Q. Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2017; pp.4700–4708.
[27] Chen C F, Gong D H, Wang H, Li Z F, Wong K Y K. Learning spatial attention for face super-resolution. IEEE Transactions on Image Processing, 2020; 30: 1219–1231.
[28] Woo S, Park J, Lee J Y, Kweon I S. Cbam: Convolutional block attention module. In Proceedings of the European conference on computer vision (ECCV), IEEE, 2018; pp.3–19.
[29] Park J, Woo S, Lee J Y, Kweon I S. Bam: Bottleneck attention module. arXiv, 2018; arXiv preprint arXiv:1807.06514.
[30] Roy A G, Navab N, Wachinger C. Concurrent spatial and channel ‘squeeze & excitation’ in fully convolutional networks. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, 2018; pp.421–429.
[31] Fu J, Liu J, Tian H J, Li Y, Bao Y J, Fang Z W, et al. Dual attention network for scene segmentation. In: Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach: IEEE, 2019; pp.3146–3154. doi: 10.1109/CVPR.2019.00326.
[32] Girshick R, Donahue J, Darrell T, Malik J. Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2014; pp.580–587.
[33] Girshick R. Fast r-cnn. In: Proceedings of the IEEE international conference on computer vision, 2015; pp.1440–1448.
[34] Ren S, He K, Girshick R, Sun J. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016; 39(6): 1137–1149.
[35] Lin T, Dollar P, Girshick R, He K, Hariharan B, Belongie S. Feature pyramid networks for object detection. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, 2019; pp.936–944.
[36] Rezatofighi H, Tsoi N, Gwak J, Sadeghian A, Reid I, Savarese S. Generalized intersection over union: A metric and a loss for bounding box regression. In: Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, 2019; pp.658–666.
[37] Cai Z, Vasconcelos N. Cascade r-cnn: Delving into high quality object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2018; pp.6154–6162.
[38] Pang J M, Chen K, Shi J P, Feng H J, Ouyang W L, Lin D H. Libra R-CNN: Towards balanced learning for object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, 2019; pp.821–830.
[39] He K M, Zhang X Y, Ren S Q, Sun J. Deep Residual Learning for Image Recognition. computer vision and pattern recognition. In: 2016 IEEE Conference on Computer Vision and Patter Recognition (CVPR), IEEE, 2016; pp.770–778.
[40] Wu T Y, Tang S, Zhang R, Cao J, Zhang Y D. Cgnet: A light-weight context guided network for semantic segmentation. IEEE Transactions on Image Processing, 2020; 30: 1169–1179.
[41] Hu J, Shen L, Sun G. Squeeze-and-excitation networks. In: Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2018; 7132–7141.
[42] open-mmlab/mmdetection. Available: https://github.com/open-mmlab/ mmdetection. Accessed on [2022-01-15].
[43] Visual Object Classes Challenge 2012 (VOC 2012). Available: http://host.robots.ox.ac.uk/pascal/VOC/voc2012. Accessed on [2022-01-15].
Downloads
Published
2022-12-27
How to Cite
Hu, Z., Yan, H., & Lou, T. (2022). Parallel channel and position attention-guided feature pyramid for pig face posture detection. International Journal of Agricultural and Biological Engineering, 15(6), 222–234. Retrieved from https://ijabe.migration.pkpps03.publicknowledgeproject.org/index.php/ijabe/article/view/7329
Issue
Section
Information Technology, Sensors and Control Systems
License
IJABE is an international peer reviewed open access journal, adopting Creative Commons Copyright Notices as follows.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
