Segmentation of field grape bunches via an improved pyramid scene parsing network

Authors

  • Shan Chen 1. College of Mechanical and Electronic Engineering, Northwest A&F University, Yangling 712100, Shaanxi, China; 2. Key Laboratory of Agricultural Internet of Things, Ministry of Agriculture and Rural Affairs, Yangling 712100, Shaanxi, China; 3. Shaanxi Key Laboratory of Agricultural Information Perception and Intelligent Services, Yangling 712100, Shaanxi, China
  • Yuyang Song College of Enology, Northwest A&F University, Yangling 712100, Shaanxi, China
  • Jinya Su School of Computer Science and Electronic Engineering, University of Essex, Colchester, CO4 3SQ, UK
  • Yulin Fang College of Enology, Northwest A&F University, Yangling 712100, Shaanxi, China
  • Lei Shen 1. College of Mechanical and Electronic Engineering, Northwest A&F University, Yangling 712100, Shaanxi, China; 2. Key Laboratory of Agricultural Internet of Things, Ministry of Agriculture and Rural Affairs, Yangling 712100, Shaanxi, China; 3. Shaanxi Key Laboratory of Agricultural Information Perception and Intelligent Services, Yangling 712100, Shaanxi, China
  • Zhiwen Mi 1. College of Mechanical and Electronic Engineering, Northwest A&F University, Yangling 712100, Shaanxi, China; 2. Key Laboratory of Agricultural Internet of Things, Ministry of Agriculture and Rural Affairs, Yangling 712100, Shaanxi, China; 3. Shaanxi Key Laboratory of Agricultural Information Perception and Intelligent Services, Yangling 712100, Shaanxi, China
  • Baofeng Su 1. College of Mechanical and Electronic Engineering, Northwest A&F University, Yangling 712100, Shaanxi, China; 2. Key Laboratory of Agricultural Internet of Things, Ministry of Agriculture and Rural Affairs, Yangling 712100, Shaanxi, China; 3. Shaanxi Key Laboratory of Agricultural Information Perception and Intelligent Services, Yangling 712100, Shaanxi, China

Keywords:

grape bunches, semantic segmentation, deep learning, improved PSPNet

Abstract

With the continuous expansion of wine grape planting areas, the mechanization and intelligence of grape harvesting have gradually become the future development trend. In order to guide the picking robot to pick grapes more efficiently in the vineyard, this study proposed a grape bunches segmentation method based on Pyramid Scene Parsing Network (PSPNet) deep semantic segmentation network for different varieties of grapes in the natural field environments. To this end, the Convolutional Block Attention Module (CBAM) attention mechanism and the atrous convolution were first embedded in the backbone feature extraction network of the PSPNet model to improve the feature extraction capability. Meanwhile, the proposed model also improved the PSPNet semantic segmentation model by fusing multiple feature layers (with more contextual information) extracted by the backbone network. The improved PSPNet was compared against the original PSPNet on a newly collected grape image dataset, and it was shown that the improved PSPNet model had an Intersection-over-Union (IoU) and Pixel Accuracy (PA) of 87.42% and 95.73%, respectively, implying an improvement of 4.36% and 9.95% over the original PSPNet model. The improved PSPNet was also compared against the state-of-the-art DeepLab-V3+ and U-Net in terms of IoU, PA, computation efficiency and robustness, and showed promising performance. It is concluded that the improved PSPNet can quickly and accurately segment grape bunches of different varieties in the natural field environments, which provides a certain technical basis for intelligent harvesting by grape picking robots. Keywords: grape bunches, semantic segmentation, deep learning, improved PSPNet DOI: 10.25165/j.ijabe.20211406.6903 Citation: Chen S, Song Y Y, Su J Y, Fang Y L, Shen L, Mi Z W, et al. Segmentation of field grape bunches via an improved pyramid scene parsing network. Int J Agric & Biol Eng, 2021; 14(6): 185–194.

References

[1] Yu Y, Zhang K, Yang L, Zhang D. Fruit detection for strawberry harvesting robot in non-structural environment based on Mask-RCNN. Computers and Electronics in Agriculture, 2019; 163: 104846. doi: 10.1016/ j.compag.2019.06.001.
[2] Xiong J, Liu Z, Lin R, Bu R, He Z L, Yang Z G, Liang C. Green grape detection and picking-point calculation in a night-time natural environment using a charge-coupled device (CCD) vision sensor with artificial illumination. Sensors (Basel, Switzerland), 2018; 18(4): 969. doi: 10.3390/s18040969.
[3] Murillo-Bracamontes E A, Martinez-Rosas M E, Miranda-Velasco M M, Martinez-Reyes H L, Martinez-Sandoval J R, Cervantes-De-Avila H. Implementation of Hough transform for fruit image segmentation. Procedia Engineering, 2012; 35: 230–239. doi: 10.1016/j.proeng.2012.04. 185.
[4] Reis M J, Morais R, Peres E, Pereira C, Contente O, Soares S, Valente A, Baptista J, Ferreira P J S, Cruz J B. Automatic detection of bunches of grapes in natural environment from color images. Journal of Applied Logic, 2012; 10(4): 285–290. doi: 10.1016/j.jal.2012.07.004.
[5] Liu S, Whitty M. Automatic grape bunch detection in vineyards with an SVM classifier. Journal of Applied Logic, 2015; 13(4): 643–653. doi: 10.1016/j.jal.2015.06.001.
[6] Pérez-Zavala R, Torres-Torriti M, Cheein F A, Troni G. A pattern recognition strategy for visual grape bunch detection in vineyards. Computers and Electronics in Agriculture, 2018; 151: 136–149. doi: 10.1016/j.compag.2018.05.019.
[7] Cecotti H, Rivera A, Farhadloo M, Pedroza M A. Grape detection with convolutional neural networks. Expert Systems with Applications, 2020; 159: 113588. doi: 10.1016/j.eswa.2020.113588.
[8] Milella A, Marani R, Petitti A, Reina G. In-field high throughput grapevine phenotyping with a consumer-grade depth camera. Computers and Electronics in Agriculture, 2019; 156: 293–306. doi: 10.1016/j.compag.2018.11.026.
[9] Marani R, Milella A, Petitti A, Reina G. Deep neural networks for grape bunch segmentation in natural images from a consumer-grade camera. Precision Agriculture, 2021; 22(2): 387–413. doi: 10.1007/s11119-020- 09736-0.
[10] Dias P A, Tabb A, Medeiros H. Multispecies fruit flower detection using a refined semantic segmentation network. IEEE robotics and automation letters, 2018; 3(4): 3003–3010.
[11] Lin K, Gong L, Huang Y, Liu C, Pan J. Deep learning-based segmentation and quantification of cucumber powdery mildew using convolutional neural network. Frontiers in plant science, 2019; 10: 155. doi: 10.3389/fpls.2019.00155.
[12] Ganchenko V, Starovoitov V, Zheng X. Image semantic segmentation based on high-resolution networks for monitoring agricultural vegetation. In: 2020 22nd International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC), IEEE, 2020; pp.264–269. doi: 10.1109/SYNASC51798.2020.0050.
[13] Tassis L M, De Souza J E T, Krohling R A. A deep learning approach combining instance and semantic segmentation to identify diseases and pests of coffee leaves from in-field images. Computers and Electronics in Agriculture, 2021; 186: 106191. doi: 10.1016/j.compag.2021.106191.
[14] Chen S, Zhang K, Zhao Y, Sun Y, Ban W, Chen Y, et al. An approach for rice bacterial leaf streak disease segmentation and disease severity estimation. Agriculture, 2021; 11(5): 420. doi: 10.3390/ agriculture11050420.
[15] Esgario J, Castro P, Tassis L M, Krohling R A. An app to assist farmers in the identification of diseases and pests of coffee leaves using deep learning. Information Processing in Agriculture, 2021; In press. doi: 10.1016/ j.inpa.2021.01.004.
[16] Kang H, Chen C J S. Fruit detection and segmentation for apple harvesting using visual sensor in orchards. Sensors, 2019; 19(20): 4599. doi: 10.3390/s19204599.
[17] Roy K, Chaudhuri S S, Pramanik S. Deep learning based real-time Industrial framework for rotten and fresh fruit detection using semantic segmentation. Microsystem Technologies, 2021; 27: 3365–3375.
[18] Li J H, Tang Y C, Zou X J, Lin G C, Wang H J. Detection of fruit-bearing branches and localization of litchi clusters for vision-based harvesting robots. IEEE Access, 2020; 8: 117746–117758.
[19] Kestur R, Meduri A, Narasipura O. MangoNet: A deep semantic segmentation architecture for a method to detect and count mangoes in an open orchard. Engineering Applications of Artificial Intelligence, 2019; 77: 59–69. doi: 10.1016/j.engappai.2018.09.011.
[20] Shu B, Mu J, Zhu Y. AMNet: Convolutional neural network embeded with attention mechanism for semantic segmentation. In: Proceedings of the 2019 3rd High Performance Computing and Cluster Technologies Conference, 2019; pp.261–266. doi: 10.1145/3341069.3342988.
[21] Lin C-Y, Chiu Y-C, Ng H F, Shih T K, Lin K-H. Global-and-local context network for semantic segmentation of street view images. Sensors, 2020; 20(10): 2907. doi: 10.3390/s20102907.
[22] Wang Y, Lyu J, Xu L, Gu Y, Zou L, Ma Z. A segmentation method for waxberry image under orchard environment. Scientia Horticulturae, 2020; 266: 109309. doi: 10.1016/j.scienta.2020.109309.
[23] Li Q, Jia W, Sun M, Hou S, Zheng Y. A novel green apple segmentation algorithm based on ensemble U-Net under complex orchard environment. Computers and Electronics in Agriculture, 2021; 180: 105900. doi: 10.1016/j.compag.2020.105900.
[24] Amiri S A, Hassanpour H. A preprocessing approach for image analysis using gamma correction. International Journal of Computer Applications, 2012; 38(12): 38–46.
[25] Woo S, Park J, Lee J-Y, Kweon I S. CBAM: Convolutional block attention module. In: Proceedings of the European conference on computer vision - ECCV 2018, Springer, Cham, 2018; pp.3–19. doi: 10.1007/978-3-030-01234-2_1.
[26] Hesamian M H, Jia W, He X, Kennedy P J. Atrous convolution for binary semantic segmentation of lung nodule. In: ICASSP 2019 - 2019 IEEE international Conference on Acoustics, Speech and Signal Processing, IEEE, 2019; pp.1015–1019. doi: 10.1109/ICASSP.2019.8682220.
[27] Fu H, Meng D, Li W, Wang Y. Bridge Crack Semantic Segmentation Based on Improved Deeplabv3+. Journal of Marine Science and Engineering, 2021; 9(6): 671. doi: 10.3390/jmse9060671.
[28] Chen Y, Li Y, Wang J, Chen W, Zhang X. Remote sensing image ship detection under complex sea conditions based on deep semantic segmentation. Remote Sensing, 2020; 12(4): 625. doi: 10.3390/ rs12040625.
[29] Wang W, Fu Y, Dong F, Li F. Semantic segmentation of remote sensing ship image via a convolutional neural networks model. IET Image Processing, 2019; 13(6): 1016–1022.
[30] Huang L, He M, Tan C, Jiang D, Li G, Yu H. Jointly network image processing: multi-task image semantic segmentation of indoor scene based on CNN. IET Image Processing, 2020; 14(15): 3689–3697. doi: 10.1049/iet-ipr.2020.0088.
[31] Pi Y, Nath N D, Behzadan A H. Detection and semantic segmentation of disaster damage in UAV footage. Journal of Computing in Civil Engineering, 2021; 35(2): 04020063. doi: 10.1061/(asce)cp.1943-5487. 0000947.

Downloads

Published

2021-12-16

How to Cite

Chen, S., Song, Y., Su, J., Fang, Y., Shen, L., Mi, Z., & Su, B. (2021). Segmentation of field grape bunches via an improved pyramid scene parsing network. International Journal of Agricultural and Biological Engineering, 14(6), 185–194. Retrieved from https://ijabe.migration.pkpps03.publicknowledgeproject.org/index.php/ijabe/article/view/6903

Issue

Vol. 14 No. 6 (2021): IJABE

Section

Information Technology, Sensors and Control Systems

License

IJABE is an international peer reviewed open access journal, adopting Creative Commons Copyright Notices as follows.


Authors who publish with this journal agree to the following terms:


  1. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.

  2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.

  3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).