Trunk detection based on laser radar and vision data fusion
Keywords:
trunk detection, data fusion, evidence theory, calibration, laser radar, vision cameraAbstract
Tree trunks detection and their location information are needed to perform effective production and management in forestry and fruit farming. A novel algorithm based on data fusion with a vision camera and a 2D laser scanner was developed to detect tree trunks accurately. The transformation was built from a laser coordinate system to an image coordinate system, and the model of a rectangle calibration plate with two inward concave regions was established to implement data alignment between two sensors data. Then, data fusion and decision with Dempster-Shafer theory were achieved through integration of decision level after designing and determining basic probability assignments of regions of interesting (RoIs) for laser and vision data respectively. Tree trunk width was calculated by using laser data to determine basic probability assignments of RoIs of laser data. And a stripping segmentation algorithm was presented to determine basic probability assignments of RoIs of vision data, by calculating the matching level of RoIs like tree trunks. A robot platform was used to acquire data from sensors and to perform the developed tree trunk detection algorithm. Combined calibration tests were conducted to calculate a conversion matrix transforming from the laser coordinate system to the image coordinate system, and then field experiments were carried out in a real pear orchard under sunny and cloudy conditions, with trunk width measurement of 120 trees and 40 images processed by the presented stripping segmentation algorithm. Results showed the algorithm was successful to detect tree trunks and data fusion improved the ability for tree trunk detection. This algorithm could provide a new method for tree trunk detection and accurate production and management in orchards. Keywords: trunk detection, data fusion, evidence theory, calibration, laser radar, vision camera DOI: 10.25165/j.ijabe.20181106.3725 Citation: Xue J L, Fan B W, Yan J, Dong S X, Ding Q S. Trunk detection based on laser radar and vision data fusion. Int J Agric & Biol Eng, 2018; 11(6): 20–26.References
[1] Gertsis A, Fountas D, Arpasanu I, Michaloudis M. Precision agriculture applications in a high density olive grove adapted for mechanical harvesting in Greece. Procedia Technology, 2013; 8(3): 152–156.
[2] Berk P, Hocevar M, Stajnko D, Belsak A. Development of alternative plant protection product application techniques in orchards, based on measurement sensing systems: A review. Computers and Electronics in Agriculture, 2016; 124: 273–288.
[3] Sanz R, Rosell J R, Llorens J, Gil E, Planas S. Relationship between tree row LiDAR-volume and leaf area density for fruit orchards and vineyards obtained with a LiDAR 3D dynamic measurement system. Agricultural and Forest Meteorology, 2013; 171-172(3): 153–162.
[4] Ježová J, Mertens L, Lambot S. Ground-penetrating radar for observing tree trunks and other cylindrical objects. Construction and Building Materials, 2016; 123: 214–225.
[5] Juman M A, Wong Y W, Rajkumar R K, Goh L J. A novel tree trunk detection method for oil-palm plantation navigation. Computers and Electronics in Agriculture, 2016; 128: 172–180.
[6] Kan J M, Li W B, Sun R S. Automatic measurement of trunk and branch diameter of standing trees based on computer vision. Proceedings of 3rd IEEE Conference on Industrial Electronics and Applications, 3-5 June, 2008; pp.995–998.
[7] Liang X, Litkey P, Hyyppä J, Kaartinen H, Kukko A, Holopainen M. Automatic plot-wise tree location mapping using single-scan terrestrial laser scanning. The Photogrammetric Journal of Finland, 2011; 22(19): 37–48.
[8] Lehtomäki M, Jaakkola A, Hyyppä J, Kukko A, Kaartinen H. Detection of vertical pole-like objects in a road environment using vehicle - based laser scanning data. Remote Sensing, 2010; 2(3): 641–664.
[9] Rahman M Z A, Gorte B. Tree filtering for high density airborne LiDAR data. International Conference on LiDAR Applications in Forest Assessment and Invertory, Edinburgh, UK, 2008.
[10] Bargoti S, Underwood J P, Nieto J I, Sukkarieh S. A pipeline for trunk localisation using LiDAR in trellis structured orchards. Field and Service Robotics, Switzerland: Springer International, 2015.
[11] Hamner B, Singh S, Bergerman M. Improving orchard efficiency with autonomous utility vehicles. 2010 ASABE Annual International Meeting, Pittsburgh, PA, USA, 2010.
[12] Guivant J, Masson F, Nebot E. Simultaneous localization and map building using natural features and absolute information. Robotics and Autonomous Systems, 2002; 40(2-3): 79–90.
[13] Zhang J, Chambers A, Maeta S, Bergerman M, Singh S. 3D perception for accurate row following: methodology and results. 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, Tokyo, Japan, 2013.
[14] Ali W, Georgsson F, Hellstrom T. Visual tree detection for autonomous navigation in forest environment. 2008 IEEE Intelligent Vehicles Symposium, Eindhoven, Netherlands, 2008.
[15] He B, Liu G, Ji Y, Si Y, Gao R. Auto recognition of navigation path for harvest robot based on machine vision. International Conference on Computer and Computing Technologies in Agriculture IV. Springer Berlin Heidelberg, 2011.
[16] Morgenroth J, Gomez C. Assessment of tree structure using a 3D image analysis technique - a proof of concept. Urban Forestry and Urban Greening, 2014; 13(1): 198–203.
[17] Cheein F A, Steiner G, Paina G P, Carelli R. Optimized EIF-SLAM algorithm for precision agriculture mapping based on stems detection. Computers and Electronics in Agriculture, 2011; 78(2): 195–207.
[18] Shalal N, Low T, Mccarthy C, Hancock N. Orchard mapping and mobile robot localisation using on-board camera and laser scanner data fusion – part B: mapping and localisation. Computers and Electronics in Agriculture, 2015; 119(Supp. 2): 267–278.
[19] Zhang S S. Navigation of an agricultural autonomous vehicle based on laser radar. MS dissertation. Nanjing: Nanjing Agricultural University, 2014; 6. 25p. (in Chinese)
[20] Liu D X. A research on LADAR-vision fusion and its application in cross-country autonomous navigation vehicle. Changsha: National University of Defense Technology, 2009; 2. 44p. (in Chinese)
[21] He Y, Wang G H, Lu D J, Peng Y N. Multisensor information fusion with application. Beijing: Publishing House of Electronics Industry, 2000. pp.28–33. (in Chinese)
[2] Berk P, Hocevar M, Stajnko D, Belsak A. Development of alternative plant protection product application techniques in orchards, based on measurement sensing systems: A review. Computers and Electronics in Agriculture, 2016; 124: 273–288.
[3] Sanz R, Rosell J R, Llorens J, Gil E, Planas S. Relationship between tree row LiDAR-volume and leaf area density for fruit orchards and vineyards obtained with a LiDAR 3D dynamic measurement system. Agricultural and Forest Meteorology, 2013; 171-172(3): 153–162.
[4] Ježová J, Mertens L, Lambot S. Ground-penetrating radar for observing tree trunks and other cylindrical objects. Construction and Building Materials, 2016; 123: 214–225.
[5] Juman M A, Wong Y W, Rajkumar R K, Goh L J. A novel tree trunk detection method for oil-palm plantation navigation. Computers and Electronics in Agriculture, 2016; 128: 172–180.
[6] Kan J M, Li W B, Sun R S. Automatic measurement of trunk and branch diameter of standing trees based on computer vision. Proceedings of 3rd IEEE Conference on Industrial Electronics and Applications, 3-5 June, 2008; pp.995–998.
[7] Liang X, Litkey P, Hyyppä J, Kaartinen H, Kukko A, Holopainen M. Automatic plot-wise tree location mapping using single-scan terrestrial laser scanning. The Photogrammetric Journal of Finland, 2011; 22(19): 37–48.
[8] Lehtomäki M, Jaakkola A, Hyyppä J, Kukko A, Kaartinen H. Detection of vertical pole-like objects in a road environment using vehicle - based laser scanning data. Remote Sensing, 2010; 2(3): 641–664.
[9] Rahman M Z A, Gorte B. Tree filtering for high density airborne LiDAR data. International Conference on LiDAR Applications in Forest Assessment and Invertory, Edinburgh, UK, 2008.
[10] Bargoti S, Underwood J P, Nieto J I, Sukkarieh S. A pipeline for trunk localisation using LiDAR in trellis structured orchards. Field and Service Robotics, Switzerland: Springer International, 2015.
[11] Hamner B, Singh S, Bergerman M. Improving orchard efficiency with autonomous utility vehicles. 2010 ASABE Annual International Meeting, Pittsburgh, PA, USA, 2010.
[12] Guivant J, Masson F, Nebot E. Simultaneous localization and map building using natural features and absolute information. Robotics and Autonomous Systems, 2002; 40(2-3): 79–90.
[13] Zhang J, Chambers A, Maeta S, Bergerman M, Singh S. 3D perception for accurate row following: methodology and results. 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, Tokyo, Japan, 2013.
[14] Ali W, Georgsson F, Hellstrom T. Visual tree detection for autonomous navigation in forest environment. 2008 IEEE Intelligent Vehicles Symposium, Eindhoven, Netherlands, 2008.
[15] He B, Liu G, Ji Y, Si Y, Gao R. Auto recognition of navigation path for harvest robot based on machine vision. International Conference on Computer and Computing Technologies in Agriculture IV. Springer Berlin Heidelberg, 2011.
[16] Morgenroth J, Gomez C. Assessment of tree structure using a 3D image analysis technique - a proof of concept. Urban Forestry and Urban Greening, 2014; 13(1): 198–203.
[17] Cheein F A, Steiner G, Paina G P, Carelli R. Optimized EIF-SLAM algorithm for precision agriculture mapping based on stems detection. Computers and Electronics in Agriculture, 2011; 78(2): 195–207.
[18] Shalal N, Low T, Mccarthy C, Hancock N. Orchard mapping and mobile robot localisation using on-board camera and laser scanner data fusion – part B: mapping and localisation. Computers and Electronics in Agriculture, 2015; 119(Supp. 2): 267–278.
[19] Zhang S S. Navigation of an agricultural autonomous vehicle based on laser radar. MS dissertation. Nanjing: Nanjing Agricultural University, 2014; 6. 25p. (in Chinese)
[20] Liu D X. A research on LADAR-vision fusion and its application in cross-country autonomous navigation vehicle. Changsha: National University of Defense Technology, 2009; 2. 44p. (in Chinese)
[21] He Y, Wang G H, Lu D J, Peng Y N. Multisensor information fusion with application. Beijing: Publishing House of Electronics Industry, 2000. pp.28–33. (in Chinese)
Downloads
Published
2018-12-08
How to Cite
Xue, J., Fan, B., Yan, J., Dong, S., & Ding, Q. (2018). Trunk detection based on laser radar and vision data fusion. International Journal of Agricultural and Biological Engineering, 11(6), 20–26. Retrieved from https://ijabe.migration.pkpps03.publicknowledgeproject.org/index.php/ijabe/article/view/3725
Issue
Section
Applied Science, Engineering and Technology
License
IJABE is an international peer reviewed open access journal, adopting Creative Commons Copyright Notices as follows.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
