Method for detecting 2D grapevine winter pruning location based on thinning algorithm and Lightweight Convolutional Neural Network
Keywords:
grapevine winter pruning, Lightweight Convolutional Neural Network, thinning algorithm, detection methodAbstract
In viticulture, there is an increasing demand for automatic winter grapevine pruning devices, for which detection of pruning location in vineyard images is a necessary task, susceptible to being automated through the use of computer vision methods. In this study, a novel 2D grapevine winter pruning location detection method was proposed for automatic winter pruning with a Y-shaped cultivation system. The method can be divided into the following four steps. First, the vineyard image was segmented by the threshold two times Red minus Green minus Blue (2R−G−B) channel and S channel; Second, extract the grapevine skeleton by Improved Enhanced Parallel Thinning Algorithm (IEPTA); Third, find the structure of each grapevine by judging the angle and distance relationship between branches; Fourth, obtain the bounding boxes from these grapevines, then pre-trained MobileNetV3_small×0.75 was utilized to classify each bounding box and finally find the pruning location. According to the detection experiment result, the method of this study achieved a precision of 98.8% and a recall of 92.3% for bud detection, an accuracy of 83.4% for pruning location detection, and a total time of 0.423 s. Therefore, the results indicated that the proposed 2D pruning location detection method had decent robustness as well as high precision that could guide automatic devices to winter prune efficiently. Keywords: grapevine winter pruning, Lightweight Convolutional Neural Network, thinning algorithm, detection method DOI: 10.25165/j.ijabe.20221503.6750 Citation: Yang Q H, Yuan Y H, Chen Y Q, Xun Y. Method for detecting 2D grapevine winter pruning location based on thinning algorithm and Lightweight Convolutional Neural Network. Int J Agric & Biol Eng, 2022; 15(3): 177–183.References
[1] Shamshiri R R, Weltzien C, Hameed I A, Yule I J, Grift T E, Balasundram S K, et al. Research and development in agricultural robotics: A perspective of digital farming. Int J Agric & Biol Eng, 2018; 11(4): 1–14.
[2] Liu S, Whitty M. Automatic grape bunch detection in vineyards with an SVM classifier. Journal of Applied Logic, 2015; 13(4): 643–653.
[3] Botterill T, Green R, Mills S. Finding a vine's structure by bottom-up parsing of cane edges. In: 2013 28th International Conference on Image and Vision Computing New Zealand (IVCNZ 2013), Wellington, New Zealand: IEEE, 2013; pp.112–117. doi: 10.1109/IVCNZ.2013.6727001.
[4] Diago M-P, Correa C, Millán B, Barreiro P, Valero C, Tardaguila J. Grapevine yield and leaf area estimation using supervised classification methodology on RGB images taken under field conditions. Sensors, 2012; 12(12): 16988–17006.
[5] Xu S, Xun Y, Jia T M, Yang Q H. Detection method for the buds on winter vines based on computer vision. In: 2014 Seventh International Symposium on Computational Intelligence and Design, Hangzhou: IEEE, 2014; pp.44–48. doi: 10.1109/ISCID.2014.26
[6] Rosenfeld A. A characterization of parallel thinning algorithms. Information and Control, 1975; 29(3): 286–291.
[7] Harris C G, Stephens M J. A combined corner and edge detector. Alvey Vision Conference, 1988; pp.147–152. doi: 10.5244/c.2.23.
[8] Pérez D S, Bromberg F, Diaz C A. Image classification for detection of winter grapevine buds in natural conditions using scale-invariant features transform, bag of features and support vector machines. Computers and Electronics in Agriculture, 2017; 135: 81–95.
[9] Lowe D G. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 2004; 60(2): 91–110.
[10] Csurka G, Dance C R, Fan L, Willamowski J, Bray C. Visual categorization with bags of keypoints. In: Workshop on Statistical Learning in Computer Vision ECCV 2004, 2004; pp.1–22.
[11] Vapnik V N, Mulier F M. Statistical learning theory. In: Learning from Data: Concepts, Theory, and Methods, IEEE, 2007; pp.99–150.
[12] Díaz C A, Pérez D S, Miatello H, Bromberg F. Grapevine buds detection and localization in 3D space based on structure from motion and 2D image classification. Computers in Industry, 2018; 99: 303–312.
[13] Zabawa L, Kicherer A, Klingbeil L, Toepfer R, Kuhlmann H, Roscher R. Counting of grapevine berries in images via semantic segmentation using convolutional neural networks. ISPRS Journal of Photogrammetry and Remote Sensing, 2020; 164: 73–83.
[14] Palacios F, Bueno G, Salido J, Diago M P, Hernandez I, Tardaguila J. Automated grapevine flower detection and quantification method based on computer vision and deep learning from on-the-go imaging using a mobile sensing platform under field conditions. Computers and Electronics in Agriculture, 2020; 178: 105796. doi: 10.1016/j.compag.2020.105796.
[15] Cruz A, Ampatzidis Y, Pierro R, Materazzi A, Panattoni A, Bellis L D, et al. Detection of grapevine yellows symptoms in Vitis vinifera L. with artificial intelligence. Computers and Electronics in Agriculture, 2019; 157: 63–76.
[16] Marset W V, Perez D S, Diaz C A, Bromberg F. Towards practical 2D grapevine bud detection with fully convolutional networks. Computers and Electronics in Agriculture, 2021; 182: 105947. doi: 10.1016/j.compag.2020.105947.
[17] Zhang T Y, Suen C Y. A fast parallel algorithm for thinning digital patterns. Communications of the ACM, 1984; 27(3): 236–239.
[18] Naccache N J, Shinghal R. An investigation into the skeletonization approach of Hilditch. Pattern Recognition, 1984; 17(3): 279–284.
[19] Lu H E, Wang P S P. A comment on “A fast parallel algorithm for thinning digital patterns”. Communications of the ACM, 1986; 29(3): 239–242.
[20] Lu H E, Wang P S P. An improved fast parallel thinning algorithm for digital patterns. In: Proceedings CVPR '85: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1985; pp.364–367.
[21] Zhao D D, Wang H B, Tao L, Zhou J. Improved EPTA parallel thinning algorithm. Computer Engineering and Applications, 2016; 52(9): 196–201. (in Chinese)
[22] Sandler M, Howard A, Zhu M L, Zhmoginov A, Chen L-C. MobileNetV2: Inverted residuals and linear bottlenecks. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA: IEEE, 2018; pp.4510–4520. doi: 10.1109/ CVPR.2018.00474.
[23] Howard A, Sandler M, Chu G, Chen L-C, Chen B, Tan M, et al. Searching for MobileNetV3. In: 2019 IEEE/CVF International Conference on Computer Vision, IEEE, 2019; pp.1314–1324.
[24] Ma N, Zhang X, Zheng H-T, Sun J. ShuffleNet V2: Practical guidelines for efficient CNN architecture design. In: Ferrari V, Hebert M, Sminchisescu C, Weiss Y (Eds). Computer Vision - ECCV 2018, Springer, 2018; pp. 122–138. doi: 10.1007/978-3-030-01264-9_8.
[25] Mingxing T, Le Q V. EfficientNet: Rethinking model scaling for Convolutional Neural Networks. arXiv, 2019; arXiv: 1905.11946 doi: 10.48550/arXiv.1905.11946.
[26] Iandola F N, Han S, Moskewicz M W, Ashraf K, Dally W J, Keutzer K. SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5 MB model size. arXiv, 2016; arXiv: 1602.07360. doi: 10.48550/arXiv.1602.07360.
[27] Rosenfeld A, Thurston M. Edge and curve detection for visual scene analysis. IEEE Transactions on Computers, 1971; C-20(5): 562–569.
[28] Panigrahi S, Nanda A, Swarnkar T. A survey on transfer learning. IEEE Transactions on Knowledge and Data Eengineering, 2010; 22(10): 1345–1359.
[29] Shorten C, Khoshgoftaar T M. A survey on image data augmentation for deep learning. Journal of Big Data, 2019; 6(1): 60. doi: 10.1186/ s40537-019-0197-0.
[30] Ren S Q, He K M, Girshick R, Sun J. Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017; 39(6): 1137–1149.
[2] Liu S, Whitty M. Automatic grape bunch detection in vineyards with an SVM classifier. Journal of Applied Logic, 2015; 13(4): 643–653.
[3] Botterill T, Green R, Mills S. Finding a vine's structure by bottom-up parsing of cane edges. In: 2013 28th International Conference on Image and Vision Computing New Zealand (IVCNZ 2013), Wellington, New Zealand: IEEE, 2013; pp.112–117. doi: 10.1109/IVCNZ.2013.6727001.
[4] Diago M-P, Correa C, Millán B, Barreiro P, Valero C, Tardaguila J. Grapevine yield and leaf area estimation using supervised classification methodology on RGB images taken under field conditions. Sensors, 2012; 12(12): 16988–17006.
[5] Xu S, Xun Y, Jia T M, Yang Q H. Detection method for the buds on winter vines based on computer vision. In: 2014 Seventh International Symposium on Computational Intelligence and Design, Hangzhou: IEEE, 2014; pp.44–48. doi: 10.1109/ISCID.2014.26
[6] Rosenfeld A. A characterization of parallel thinning algorithms. Information and Control, 1975; 29(3): 286–291.
[7] Harris C G, Stephens M J. A combined corner and edge detector. Alvey Vision Conference, 1988; pp.147–152. doi: 10.5244/c.2.23.
[8] Pérez D S, Bromberg F, Diaz C A. Image classification for detection of winter grapevine buds in natural conditions using scale-invariant features transform, bag of features and support vector machines. Computers and Electronics in Agriculture, 2017; 135: 81–95.
[9] Lowe D G. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 2004; 60(2): 91–110.
[10] Csurka G, Dance C R, Fan L, Willamowski J, Bray C. Visual categorization with bags of keypoints. In: Workshop on Statistical Learning in Computer Vision ECCV 2004, 2004; pp.1–22.
[11] Vapnik V N, Mulier F M. Statistical learning theory. In: Learning from Data: Concepts, Theory, and Methods, IEEE, 2007; pp.99–150.
[12] Díaz C A, Pérez D S, Miatello H, Bromberg F. Grapevine buds detection and localization in 3D space based on structure from motion and 2D image classification. Computers in Industry, 2018; 99: 303–312.
[13] Zabawa L, Kicherer A, Klingbeil L, Toepfer R, Kuhlmann H, Roscher R. Counting of grapevine berries in images via semantic segmentation using convolutional neural networks. ISPRS Journal of Photogrammetry and Remote Sensing, 2020; 164: 73–83.
[14] Palacios F, Bueno G, Salido J, Diago M P, Hernandez I, Tardaguila J. Automated grapevine flower detection and quantification method based on computer vision and deep learning from on-the-go imaging using a mobile sensing platform under field conditions. Computers and Electronics in Agriculture, 2020; 178: 105796. doi: 10.1016/j.compag.2020.105796.
[15] Cruz A, Ampatzidis Y, Pierro R, Materazzi A, Panattoni A, Bellis L D, et al. Detection of grapevine yellows symptoms in Vitis vinifera L. with artificial intelligence. Computers and Electronics in Agriculture, 2019; 157: 63–76.
[16] Marset W V, Perez D S, Diaz C A, Bromberg F. Towards practical 2D grapevine bud detection with fully convolutional networks. Computers and Electronics in Agriculture, 2021; 182: 105947. doi: 10.1016/j.compag.2020.105947.
[17] Zhang T Y, Suen C Y. A fast parallel algorithm for thinning digital patterns. Communications of the ACM, 1984; 27(3): 236–239.
[18] Naccache N J, Shinghal R. An investigation into the skeletonization approach of Hilditch. Pattern Recognition, 1984; 17(3): 279–284.
[19] Lu H E, Wang P S P. A comment on “A fast parallel algorithm for thinning digital patterns”. Communications of the ACM, 1986; 29(3): 239–242.
[20] Lu H E, Wang P S P. An improved fast parallel thinning algorithm for digital patterns. In: Proceedings CVPR '85: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1985; pp.364–367.
[21] Zhao D D, Wang H B, Tao L, Zhou J. Improved EPTA parallel thinning algorithm. Computer Engineering and Applications, 2016; 52(9): 196–201. (in Chinese)
[22] Sandler M, Howard A, Zhu M L, Zhmoginov A, Chen L-C. MobileNetV2: Inverted residuals and linear bottlenecks. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA: IEEE, 2018; pp.4510–4520. doi: 10.1109/ CVPR.2018.00474.
[23] Howard A, Sandler M, Chu G, Chen L-C, Chen B, Tan M, et al. Searching for MobileNetV3. In: 2019 IEEE/CVF International Conference on Computer Vision, IEEE, 2019; pp.1314–1324.
[24] Ma N, Zhang X, Zheng H-T, Sun J. ShuffleNet V2: Practical guidelines for efficient CNN architecture design. In: Ferrari V, Hebert M, Sminchisescu C, Weiss Y (Eds). Computer Vision - ECCV 2018, Springer, 2018; pp. 122–138. doi: 10.1007/978-3-030-01264-9_8.
[25] Mingxing T, Le Q V. EfficientNet: Rethinking model scaling for Convolutional Neural Networks. arXiv, 2019; arXiv: 1905.11946 doi: 10.48550/arXiv.1905.11946.
[26] Iandola F N, Han S, Moskewicz M W, Ashraf K, Dally W J, Keutzer K. SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5 MB model size. arXiv, 2016; arXiv: 1602.07360. doi: 10.48550/arXiv.1602.07360.
[27] Rosenfeld A, Thurston M. Edge and curve detection for visual scene analysis. IEEE Transactions on Computers, 1971; C-20(5): 562–569.
[28] Panigrahi S, Nanda A, Swarnkar T. A survey on transfer learning. IEEE Transactions on Knowledge and Data Eengineering, 2010; 22(10): 1345–1359.
[29] Shorten C, Khoshgoftaar T M. A survey on image data augmentation for deep learning. Journal of Big Data, 2019; 6(1): 60. doi: 10.1186/ s40537-019-0197-0.
[30] Ren S Q, He K M, Girshick R, Sun J. Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017; 39(6): 1137–1149.
Downloads
Published
2022-06-30
How to Cite
Yang, Q., Yuan, Y., Chen, Y., & Xun, Y. (2022). Method for detecting 2D grapevine winter pruning location based on thinning algorithm and Lightweight Convolutional Neural Network. International Journal of Agricultural and Biological Engineering, 15(3), 177–183. Retrieved from https://ijabe.migration.pkpps03.publicknowledgeproject.org/index.php/ijabe/article/view/6750
Issue
Section
Information Technology, Sensors and Control Systems
License
IJABE is an international peer reviewed open access journal, adopting Creative Commons Copyright Notices as follows.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
