Preprocessing method of night vision image application in apple harvesting robot
Keywords:
apple harvesting robot, night vision image, preprocessing method, color analysis, noise analysisAbstract
Due to the low working efficiency of apple harvesting robots, there is still a long way to go for commercialization. The machine performance and extended operating time are the two research aspects for improving efficiencies of harvesting robots, this study focused on the extended operating time and proposed a round-the-clock operation mode. Due to the influences of light, temperature, humidity, etc., the working environment at night is relatively complex, and thus restricts the operating efficiency of the apple harvesting robot. Three different artificial light sources (incandescent lamp, fluorescent lamp, and LED lights) were selected for auxiliary light according to certain rules so that the apple night vision images could be captured. In addition, by color analysis, night and natural light images were compared to find out the color characteristics of the night vision images, and intuitive visual and difference image methods were used to analyze the noise characteristics. The results showed that the incandescent lamp is the best artificial auxiliary light for apple harvesting robots working at night, and the type of noise contained in apple night vision images is Gaussian noise mixed with some salt and pepper noise. The preprocessing method can provide a theoretical and technical reference for subsequent image processing. Keywords: apple harvesting robot, night vision image, preprocessing method, color analysis, noise analysis DOI: 10.25165/j.ijabe.20181102.2822 Citation: Jia W K, Zheng Y J, Zhao D A, Yin X, Liu X Y, Du R C. Preprocessing method of night vision image application in apple harvesting robot. Int J Agric & Biol Eng, 2018; 11(2): 158–163.References
[1] Pedersen S M, Fountas S, Have H, Blackmore B S. Agricultural robots - system analysis and economic feasibility. Precision agriculture, 2006; 7(4): 295–308.
[2] Zhang L B, Yang Q H, Bao G J, Wang Y, Qi L Y, Gao F, et al. Overview of research on agricultural robots in China. Int J Agric & Biol Eng, 2008; 1(1): 12–21.
[3] Van Henten E J, Van’t Slot D A, Hol C W J, van Willigenburg L G. Optimal manipulator design for a cucumber harvesting robot. Computers and Electronics in Agriculture, 2009; 65(2): 247–257.
[4] Bac C W, Van Henten E J, Hemming J, Edan Y. Harvesting robots for high-value crops: state-of-the-art review and challenges ahead. Journal of Field Robotics, 2014; 31(6): 888–911.
[5] Fernández R, Salinas C, Montes H, Sarria J. Multisensory system for fruit harvesting robots. Experimental testing in natural scenarios and with different kinds of crops. Sensors, 2014; 14(12): 23885–23904.
[6] Wang L L, Zhao B, Fan J W, Hu X A, Wei S, Li Y S, et al. Development of a tomato harvesting robot used in greenhouse. Int J Agric & Biol Eng, 2017; 10(4): 140–149.
[7] Feng Q C, Zou W, Fan P F, Zhang C F, Wang X. Design and test of robotic harvesting system for cherry tomato. Int J Agric & Biol Eng, 2018; 11(1): 96–100.
[8] Baeten J, Donné K, Boedrij S, Beckers W, Claesen E. Autonomous fruit picking machine: a robotic apple harvester. Springer Tracts in Advanced Robotics, 2008; 42: 531–539.
[9] Zhao D A, Lv J D, Ji W, Zhang Y, Chen Y. Design and control of an apple harvesting robot. Biosystems Engineering, 2011; 110(2): 112–122.
[10] Ji W, Zhao D A, Cheng F Y, Xu B, Zhang Y, Wang J J. Automatic recognition vision system guided for apple harvesting robot. Computers and Electrical Engineering, 2012; 38(5): 1186–1195.
[11] Xiao C Y, Zheng L H, Li M Z, Ma C Y. Apple detection from apple tree image based on BP neural network and Hough transform. Int J Agric & Biol Eng, 2015; 8(6): 46–53.
[12] Cai J R, Sun H B, Li Y P, Sun L, Lu H. Fruit trees 3-D information perception and reconstruction based on binocular stereo vision. Transactions of the CSAM, 2012; 43(3): 152–156. (in Chinese)
[13] Zhao D A, Shen T, Chen Y, Jia W K. Fast tracking and recognition of overlapping fruit for apple harvesting robot. Transactions of the CSAE, 2015; 31(2): 22–28. (in Chinese)
[14] Ji W, Cheng F Y, Zhao D A, Lü J. Obstacle avoidance method of apple harvesting robot manipulator. Transactions of the CSAM, 2013; 44(11): 253–259. (in Chinese)
[15] Yuan Y W, Zhang X C, Hu X A. Algorithm for optimization of apple harvesting path and simulation. Transactions of the CSAE, 2009; 25(4): 141–144. (in Chinese)
[16] Li B R, Long Y, Song H B. Detection of green apples in natural scenes based on saliency theory and Gaussian curve fitting. Int J Agric & Biol Eng, 2018; 11(1): 192–198.
[17] Payne A B, Walsh K B, Subedi P P, Jarvis D. Estimating mango crop yield using image analysis using fruit at ‘stone hardening’ stage and night time imaging. Computers and Electronics in Agriculture, 2014; 100: 160–167.
[18] Font D, Pallejà T, Tresanchez M, Teixidó M, Martinez D, Moreno J, et al. Counting red grapes in vineyards by detecting specular spherical reflection peaks in RGB images obtained at night with artificial illumination. Computers and Electronics in Agriculture, 2014; 108: 105–111.
[19] Guo F, Cao Q X, Cui Y J, Masateru N. Fruit location and stem detection method for strawberry harvesting robot. Transactions of the CSAE, 2008; 24(10): 89–94.
[20] Hayashi S, Shigematsu K, Yamamoto S, Kurita M. Evaluation of a strawberry-harvesting robot in a field test. Biosystems Engineering, 2010; 105: 160–171.
[21] Zhang C L, Zhang J, Zhang J X, Li W. Recognition of green apple in similar background. Transactions of the CSAM, 2014; 45(10): 277–281. (in Chinese).
[22] Liu X Y, Zhao D A, Jia W K, Ruan C Z, Tang S P, Shen T. A method of segmenting apples at night based on color and position information. Computers and Electronics in Agriculture, 2016; 122: 118–123.
[23] Jia W K, Zhao D A, Ruan C Z, Liu X Y, Chen Y Y, Ji W. Combination method of night vision image denoising based on wavelet transform and ICA. Transactions of the CSAM, 2015; 46(9): 9–17. (in Chinese)
[24] Han J, Yue J, Zhang Y, Bai L F. Salient contour extraction from complex natural scene in night vision image. Infrared Physics & Technology, 2014; 63: 165–177.
[25] Talebi H, Milanfar P. Global image denoising. IEEE Transactions on Image Processing, 2014; 23(2): 755–768.
[26] Zhao M, Zhang H, Sun J. A novel image retrieval method based on multi-trend structure descriptor. Journal of Visual Communication & Image Representation, 2016; 38(C): 73–81.
[27] Hou S J, Chen L, Tao D C, Zheng Y J. Multi-layer multi-view topic model for classifying advertising video. Pattern Recognition, 2017; 68: 66–81.
[28] Meng L L, Liang J, Samarawickrama U, Zhao Y, Bai H H, Kaup A. Multiple description coding with randomly and uniformly offset quantizers. IEEE Transactions on Image Processing, 2014, 23(2): 582–595.
[29] Ding S H, Levant A, Li S H. Simple homogeneous sliding-mode controller. Automatica, 2016; 67: 22–32.
[30] Liu X M, Zhang K J, Li S T, Wei H K. Optimal control of switching times in switched stochastic systems. Asian Journal of Control, 2015; 17(5): 1580–1589.
[2] Zhang L B, Yang Q H, Bao G J, Wang Y, Qi L Y, Gao F, et al. Overview of research on agricultural robots in China. Int J Agric & Biol Eng, 2008; 1(1): 12–21.
[3] Van Henten E J, Van’t Slot D A, Hol C W J, van Willigenburg L G. Optimal manipulator design for a cucumber harvesting robot. Computers and Electronics in Agriculture, 2009; 65(2): 247–257.
[4] Bac C W, Van Henten E J, Hemming J, Edan Y. Harvesting robots for high-value crops: state-of-the-art review and challenges ahead. Journal of Field Robotics, 2014; 31(6): 888–911.
[5] Fernández R, Salinas C, Montes H, Sarria J. Multisensory system for fruit harvesting robots. Experimental testing in natural scenarios and with different kinds of crops. Sensors, 2014; 14(12): 23885–23904.
[6] Wang L L, Zhao B, Fan J W, Hu X A, Wei S, Li Y S, et al. Development of a tomato harvesting robot used in greenhouse. Int J Agric & Biol Eng, 2017; 10(4): 140–149.
[7] Feng Q C, Zou W, Fan P F, Zhang C F, Wang X. Design and test of robotic harvesting system for cherry tomato. Int J Agric & Biol Eng, 2018; 11(1): 96–100.
[8] Baeten J, Donné K, Boedrij S, Beckers W, Claesen E. Autonomous fruit picking machine: a robotic apple harvester. Springer Tracts in Advanced Robotics, 2008; 42: 531–539.
[9] Zhao D A, Lv J D, Ji W, Zhang Y, Chen Y. Design and control of an apple harvesting robot. Biosystems Engineering, 2011; 110(2): 112–122.
[10] Ji W, Zhao D A, Cheng F Y, Xu B, Zhang Y, Wang J J. Automatic recognition vision system guided for apple harvesting robot. Computers and Electrical Engineering, 2012; 38(5): 1186–1195.
[11] Xiao C Y, Zheng L H, Li M Z, Ma C Y. Apple detection from apple tree image based on BP neural network and Hough transform. Int J Agric & Biol Eng, 2015; 8(6): 46–53.
[12] Cai J R, Sun H B, Li Y P, Sun L, Lu H. Fruit trees 3-D information perception and reconstruction based on binocular stereo vision. Transactions of the CSAM, 2012; 43(3): 152–156. (in Chinese)
[13] Zhao D A, Shen T, Chen Y, Jia W K. Fast tracking and recognition of overlapping fruit for apple harvesting robot. Transactions of the CSAE, 2015; 31(2): 22–28. (in Chinese)
[14] Ji W, Cheng F Y, Zhao D A, Lü J. Obstacle avoidance method of apple harvesting robot manipulator. Transactions of the CSAM, 2013; 44(11): 253–259. (in Chinese)
[15] Yuan Y W, Zhang X C, Hu X A. Algorithm for optimization of apple harvesting path and simulation. Transactions of the CSAE, 2009; 25(4): 141–144. (in Chinese)
[16] Li B R, Long Y, Song H B. Detection of green apples in natural scenes based on saliency theory and Gaussian curve fitting. Int J Agric & Biol Eng, 2018; 11(1): 192–198.
[17] Payne A B, Walsh K B, Subedi P P, Jarvis D. Estimating mango crop yield using image analysis using fruit at ‘stone hardening’ stage and night time imaging. Computers and Electronics in Agriculture, 2014; 100: 160–167.
[18] Font D, Pallejà T, Tresanchez M, Teixidó M, Martinez D, Moreno J, et al. Counting red grapes in vineyards by detecting specular spherical reflection peaks in RGB images obtained at night with artificial illumination. Computers and Electronics in Agriculture, 2014; 108: 105–111.
[19] Guo F, Cao Q X, Cui Y J, Masateru N. Fruit location and stem detection method for strawberry harvesting robot. Transactions of the CSAE, 2008; 24(10): 89–94.
[20] Hayashi S, Shigematsu K, Yamamoto S, Kurita M. Evaluation of a strawberry-harvesting robot in a field test. Biosystems Engineering, 2010; 105: 160–171.
[21] Zhang C L, Zhang J, Zhang J X, Li W. Recognition of green apple in similar background. Transactions of the CSAM, 2014; 45(10): 277–281. (in Chinese).
[22] Liu X Y, Zhao D A, Jia W K, Ruan C Z, Tang S P, Shen T. A method of segmenting apples at night based on color and position information. Computers and Electronics in Agriculture, 2016; 122: 118–123.
[23] Jia W K, Zhao D A, Ruan C Z, Liu X Y, Chen Y Y, Ji W. Combination method of night vision image denoising based on wavelet transform and ICA. Transactions of the CSAM, 2015; 46(9): 9–17. (in Chinese)
[24] Han J, Yue J, Zhang Y, Bai L F. Salient contour extraction from complex natural scene in night vision image. Infrared Physics & Technology, 2014; 63: 165–177.
[25] Talebi H, Milanfar P. Global image denoising. IEEE Transactions on Image Processing, 2014; 23(2): 755–768.
[26] Zhao M, Zhang H, Sun J. A novel image retrieval method based on multi-trend structure descriptor. Journal of Visual Communication & Image Representation, 2016; 38(C): 73–81.
[27] Hou S J, Chen L, Tao D C, Zheng Y J. Multi-layer multi-view topic model for classifying advertising video. Pattern Recognition, 2017; 68: 66–81.
[28] Meng L L, Liang J, Samarawickrama U, Zhao Y, Bai H H, Kaup A. Multiple description coding with randomly and uniformly offset quantizers. IEEE Transactions on Image Processing, 2014, 23(2): 582–595.
[29] Ding S H, Levant A, Li S H. Simple homogeneous sliding-mode controller. Automatica, 2016; 67: 22–32.
[30] Liu X M, Zhang K J, Li S T, Wei H K. Optimal control of switching times in switched stochastic systems. Asian Journal of Control, 2015; 17(5): 1580–1589.
Downloads
Published
2018-03-31
How to Cite
Jia, W., Zheng, Y., Zhao, D., Yin, X., Liu, X., & Du, R. (2018). Preprocessing method of night vision image application in apple harvesting robot. International Journal of Agricultural and Biological Engineering, 11(2), 158–163. Retrieved from https://ijabe.migration.pkpps03.publicknowledgeproject.org/index.php/ijabe/article/view/2822
Issue
Section
Information Technology, Sensors and Control Systems
License
IJABE is an international peer reviewed open access journal, adopting Creative Commons Copyright Notices as follows.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
