Accurate and rapid image segmentation method for bayberry automatic picking via machine learning
Keywords:
bayberry, image segmentation, machine learning, automatic picking, computer visionAbstract
Due to the short ripening period and complex picking environment, bayberry generally relies on mechanical equipment for picking, especially the automatic picking system guided by vision. Thus, it is crucial to locate the bayberry in the view accurately and rapidly. Although efforts have been made, the existing methods are difficult to implement due to the limited amount of data and the processing speed. In this study, an accurate and rapid segmentation method based on machine learning was proposed to address this problem. First, the images collected by the visual guidance system were pre-processed by contrast-limited adaptive histogram equalization (CLAHE) based on the Y component of the YUV color space. Taking advantage of the color difference map of RB and RG for the segmentation of different colors, an adaptive color difference map foreground segmentation method was then adopted for bayberry region foreground segmentation. Finally, distance transforms and marking control watershed methods were exploited to achieve single bayberry fruit segmentation. Furthermore, with the help of the convex hull theory and fruit shape characteristics, the irregular background interference areas were filtered out, which improved the accuracy of bayberry segmentation performance. The experimental results show that this method can achieve better segmentation of bayberry in complex orchard environment with an accuracy of 97.4% and only takes 0.136 s to calculate once. Key words: bayberry, image segmentation, machine learning, automatic picking, computer vision DOI: 10.25165/j.ijabe.20231606.6834 Citation: Lei H, Li C T, Tang Y, Zhong Z Y, Jiao Z Y. Accurate and rapid image segmentation method for bayberry automatic picking via machine learning. Int J Agric & Biol Eng, 2023; 16(6): 246–254.References
[1] Qiu Y Y, Shen B, Guo X Z, Xie X B, Zheng X L, Qi X J, et al. Effects of foliar nutrition on leaf photosynthesis and fruit quality of Chinese bayberry. Journal of Fruit Science, 2009; 26(6): 902–906. (in Chinese)
[2] Fang Z X, Bhandari, B. Comparing the efficiency of protein and maltodextrin on spray drying of bayberry juice. Food Research International, 2012; 48(2): 478–483.
[3] Linker R, Cohen O, Naor A. Determination of the number of green apples in RGB images recorded in orchards. Computers and Electronics in Agriculture, 2012; 81: 45–57.
[4] Wei X Q, Jia K, Lan J H, Li Y W, Zeng Y L, Wang C M. Automatic method of fruit object extraction under complex agricultural background for vision system of fruit picking robot. Optik, 2014; 125(19): 5684–5689.
[5] Wang C L, Tang Y C, Zou X J, SiTu W M, Feng W X. A robust fruit image segmentation algorithm against varying illumination for vision system of fruit harvesting robot. Optik, 2017; 131: 626–631.
[6] Xu L M, Lyu J D. Recognition method for apple fruit based on SUSAN and PCNN. Multimedia Tools and Applications, 2018; 77: 7205–7219.
[7] Septiarini A, Hamdani H, Hatta H R, Anwar K. Automatic image segmentation of oil palm fruits by applying the contour-based approach. Scientia Horticulturae, 2020; 261: 108939.
[8] Lu J, Lee W S, Gan H, Hu X W. Immature citrus fruit detection based on local binary pattern feature and hierarchical contour analysis. Biosystems Engineering, 2018; 171: 78–90.
[9] Si Y S, Liu G, Feng J. Location of apples in trees using stereoscopic vision. Computers and Electronics in Agriculture, 2015; 112: 68–74.
[10] Xu L M. , He K R, Lyu J D. Bayberry image segmentation based on manifold ranking salient object detection method. Biosystems Engineering, 2019; 178: 264–274.
[11] He Z L, Xiong J T, Chen S M, Li Z X, Chen S F, Zhong Z, et al. A method of green citrus detection based on a deep bounding box regression forest. Biosystems Engineering, 2020; 193: 206–215.
[12] Bargoti S, Underwood J P. Image segmentation for fruit detection and yield estimation in apple orchards. Journal of Field Robotics, 2017; 34(6): 1039–1060.
[13] Li Q W, Jia W K, Sun M L, Hou S J, Zheng Y J. A novel green apple segmentation algorithm based on ensemble U-Net under complex orchard environment. Computers and Electronics in Agriculture, 2021; 180: 105900.
[14] Yu Y, Zhang K L, Yang L, Zhang D X. Fruit detection for strawberry harvesting robot in non-structural environment based on Mask-RCNN. Computers and Electronics in Agriculture, 2019; 163: 104846.
[15] Flores P, Zhang Z, Igathinathane C, Jithin M, Naik D, Stenger J, et al. Distinguishing seedling volunteer corn from soybean through greenhouse color, color-infrared, and fused images using machine and deep learning. Industrial Crops and Products, 2021; 161: 113223.
[16] Reza A M. Realization of the contrast limited adaptive histogram equalization (CLAHE) for real-time image enhancement. Journal of VLSI Signal Processing Systems for Signal, Image and Video Technology, 2004; 38(1): 35–44.
[17] Otsu N. A threshold selection method from gray-level histograms. IEEE Transactions on Systems, Man, and Cybernetics, 1979; 9(1): 62–66.
[18] Burtsev S V, Kuzmin Y P. An efficient flood-filling algorithm. Computers & Graphics, 1993; 17(5): 549–561.
[19] Dorj U O, Lee M, Yun S S. An yield estimation in citrus orchards via fruit detection and counting using image processing. Computers and Electronics in Agriculture, 2017; 140: 103–112.
[20] Jung C, Kim C. Segmenting clustered nuclei using H-minima transform-based marker extraction and contour parameterization. IEEE Transactions on Biomedical Engineering, 2010; 57(10): 2600–2604.
[21] Wang D D, Song H B, Tie Z H, Zhang W Y, He D J. Recognition and localization of occluded apples using K-means clustering algorithm and convex hull theory: A comparison. Multimedia Tools and Applications, 2016; 75(6): 3177–3198.
[22] Lyu J D, Wang F, Xu L M, Ma Z H, Yang B. A segmentation method of bagged green apple image. Scientia Horticulturae, 2019; 246: 411–417.
[23] Guo Q W, Chen Y Y, Tang Y, Zhuang J J, He Y, Hou C J, et al. Lychee fruit detection based on monocular machine vision in orchard environment. Sensors, 2019; 19(19): 4091.
[24] Zhao D A, Liu X Y, Chen Y, Ji W, Jia W K, Hu C L. Image recognition at night for apple picking robot. Transactions of the CSAM, 2015; 46(3): 15–22.
[25] Fu L S, Wang B, Cui Y J, Su S, Yoshinori G, Kobayashi T. Kiwifruit recognition at nighttime using artificial lighting based on machine vision. In J Agric & Biol Eng, 2015; 8(4): 52–59.
[26] Zhuang Z M, Li N, Joseph Raj A. N, Mahesh V G V, Qiu S M. An RDAU-NET model for lesion segmentation in breast ultrasound images. PloS One, 2019; 14(8): e0221535.
[27] Zhuang J J. Luo S M, Hou C J, Tang Y, He Y, Xue X Y. Detection of orchard citrus fruits using a monocular machine vision-based method for automatic fruit picking applications. Computers and Electronics in Agriculture, 2018; 152: 64–73.
[2] Fang Z X, Bhandari, B. Comparing the efficiency of protein and maltodextrin on spray drying of bayberry juice. Food Research International, 2012; 48(2): 478–483.
[3] Linker R, Cohen O, Naor A. Determination of the number of green apples in RGB images recorded in orchards. Computers and Electronics in Agriculture, 2012; 81: 45–57.
[4] Wei X Q, Jia K, Lan J H, Li Y W, Zeng Y L, Wang C M. Automatic method of fruit object extraction under complex agricultural background for vision system of fruit picking robot. Optik, 2014; 125(19): 5684–5689.
[5] Wang C L, Tang Y C, Zou X J, SiTu W M, Feng W X. A robust fruit image segmentation algorithm against varying illumination for vision system of fruit harvesting robot. Optik, 2017; 131: 626–631.
[6] Xu L M, Lyu J D. Recognition method for apple fruit based on SUSAN and PCNN. Multimedia Tools and Applications, 2018; 77: 7205–7219.
[7] Septiarini A, Hamdani H, Hatta H R, Anwar K. Automatic image segmentation of oil palm fruits by applying the contour-based approach. Scientia Horticulturae, 2020; 261: 108939.
[8] Lu J, Lee W S, Gan H, Hu X W. Immature citrus fruit detection based on local binary pattern feature and hierarchical contour analysis. Biosystems Engineering, 2018; 171: 78–90.
[9] Si Y S, Liu G, Feng J. Location of apples in trees using stereoscopic vision. Computers and Electronics in Agriculture, 2015; 112: 68–74.
[10] Xu L M. , He K R, Lyu J D. Bayberry image segmentation based on manifold ranking salient object detection method. Biosystems Engineering, 2019; 178: 264–274.
[11] He Z L, Xiong J T, Chen S M, Li Z X, Chen S F, Zhong Z, et al. A method of green citrus detection based on a deep bounding box regression forest. Biosystems Engineering, 2020; 193: 206–215.
[12] Bargoti S, Underwood J P. Image segmentation for fruit detection and yield estimation in apple orchards. Journal of Field Robotics, 2017; 34(6): 1039–1060.
[13] Li Q W, Jia W K, Sun M L, Hou S J, Zheng Y J. A novel green apple segmentation algorithm based on ensemble U-Net under complex orchard environment. Computers and Electronics in Agriculture, 2021; 180: 105900.
[14] Yu Y, Zhang K L, Yang L, Zhang D X. Fruit detection for strawberry harvesting robot in non-structural environment based on Mask-RCNN. Computers and Electronics in Agriculture, 2019; 163: 104846.
[15] Flores P, Zhang Z, Igathinathane C, Jithin M, Naik D, Stenger J, et al. Distinguishing seedling volunteer corn from soybean through greenhouse color, color-infrared, and fused images using machine and deep learning. Industrial Crops and Products, 2021; 161: 113223.
[16] Reza A M. Realization of the contrast limited adaptive histogram equalization (CLAHE) for real-time image enhancement. Journal of VLSI Signal Processing Systems for Signal, Image and Video Technology, 2004; 38(1): 35–44.
[17] Otsu N. A threshold selection method from gray-level histograms. IEEE Transactions on Systems, Man, and Cybernetics, 1979; 9(1): 62–66.
[18] Burtsev S V, Kuzmin Y P. An efficient flood-filling algorithm. Computers & Graphics, 1993; 17(5): 549–561.
[19] Dorj U O, Lee M, Yun S S. An yield estimation in citrus orchards via fruit detection and counting using image processing. Computers and Electronics in Agriculture, 2017; 140: 103–112.
[20] Jung C, Kim C. Segmenting clustered nuclei using H-minima transform-based marker extraction and contour parameterization. IEEE Transactions on Biomedical Engineering, 2010; 57(10): 2600–2604.
[21] Wang D D, Song H B, Tie Z H, Zhang W Y, He D J. Recognition and localization of occluded apples using K-means clustering algorithm and convex hull theory: A comparison. Multimedia Tools and Applications, 2016; 75(6): 3177–3198.
[22] Lyu J D, Wang F, Xu L M, Ma Z H, Yang B. A segmentation method of bagged green apple image. Scientia Horticulturae, 2019; 246: 411–417.
[23] Guo Q W, Chen Y Y, Tang Y, Zhuang J J, He Y, Hou C J, et al. Lychee fruit detection based on monocular machine vision in orchard environment. Sensors, 2019; 19(19): 4091.
[24] Zhao D A, Liu X Y, Chen Y, Ji W, Jia W K, Hu C L. Image recognition at night for apple picking robot. Transactions of the CSAM, 2015; 46(3): 15–22.
[25] Fu L S, Wang B, Cui Y J, Su S, Yoshinori G, Kobayashi T. Kiwifruit recognition at nighttime using artificial lighting based on machine vision. In J Agric & Biol Eng, 2015; 8(4): 52–59.
[26] Zhuang Z M, Li N, Joseph Raj A. N, Mahesh V G V, Qiu S M. An RDAU-NET model for lesion segmentation in breast ultrasound images. PloS One, 2019; 14(8): e0221535.
[27] Zhuang J J. Luo S M, Hou C J, Tang Y, He Y, Xue X Y. Detection of orchard citrus fruits using a monocular machine vision-based method for automatic fruit picking applications. Computers and Electronics in Agriculture, 2018; 152: 64–73.
Downloads
Published
2024-02-06
How to Cite
Lei, H., Li, C., Tang, Y., Zhong, Z., & Jiao, Z. (2024). Accurate and rapid image segmentation method for bayberry automatic picking via machine learning. International Journal of Agricultural and Biological Engineering, 16(6), 246–254. Retrieved from https://ijabe.migration.pkpps03.publicknowledgeproject.org/index.php/ijabe/article/view/6834
Issue
Section
Information Technology, Sensors and Control Systems
License
IJABE is an international peer reviewed open access journal, adopting Creative Commons Copyright Notices as follows.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
