Recognition of field roads based on improved U-Net++ Network

Authors

  • Lili Yang College of Information and Electrical Engineering, China Agricultural University, Beijing 100083, China
  • Yuanbo Li College of Information and Electrical Engineering, China Agricultural University, Beijing 100083, China
  • Mengshuai Chang College of Information and Electrical Engineering, China Agricultural University, Beijing 100083, China
  • Yuanyuan Xu College of Information and Electrical Engineering, China Agricultural University, Beijing 100083, China
  • Bingbing Hu Kunlun Beidou Technologies Co. LTD., Beijing 102200, China
  • Xinxin Wang College of Information and Electrical Engineering, China Agricultural University, Beijing 100083, China
  • Caicong Wu College of Information and Electrical Engineering, China Agricultural University, Beijing 100083, China

Keywords:

image segmentation, unmanned agricultural machinery, field roads, point cloud super-resolution, point cloud bird's eye view

Abstract

Unmanned driving of agricultural machinery has garnered significant attention in recent years, especially with the development of precision farming and sensor technologies. To achieve high performance and low cost, perception tasks are of great importance. In this study, a low-cost and high-safety method was proposed for field road recognition in unmanned agricultural machinery. The approach of this study utilized point clouds, with low-resolution lidar point clouds as inputs, generating high-resolution point clouds and Bird's Eye View (BEV) images that were encoded with several basic statistics. Using a BEV representation, road detection was reduced to a single-scale problem that could be addressed with an improved U-Net++ neural network. Three enhancements were proposed for U-Net++: 1) replacing the convolutional kernel in the original U-Net++ with an Asymmetric Convolution Block (ACBlock); 2) adding a multi-branch Asymmetric Dilated Convolutional Block (MADC) in the highest semantic information layer; 3) adding an Attention Gate (AG) model to the long-skip-connection in the decoding stage. The results of experiments of this study showed that our algorithm achieved a Mean Intersection Over Union of 96.54% on the 16-channel point clouds, which was 7.35 percentage points higher than U-Net++. Furthermore, the average processing time of the model was about 70 ms, meeting the time requirements of unmanned driving in agricultural machinery. The proposed method of this study can be applied to enhance the perception ability of unmanned agricultural machinery thereby increasing the safety of field road driving. Keywords: image segmentation, unmanned agricultural machinery, field roads, point cloud super-resolution, point cloud bird's eye view DOI: 10.25165/j.ijabe.20231602.7941 Citation: Yang L L, Li Y B, Chang M S, Xu Y Y, Hu B B, Wang X X, et al. Recognition of field roads based on improved U-Net++ Network. Int J Agric & Biol Eng, 2023; 16(2): 171-178.

References

[1] Cui X Z, Feng Q, Wang S Z, Zhang J H. Monocular depth estimation with self-supervised learning for vineyard unmanned agricultural vehicle. Sensors, 2022; 22(3): 721. doi: 10.3390/s22030721.
[2] Zhu N Y, Liu X, Liu Z Q, Hu K, Wang Y K, Tan J L, et al. Deep learning for smart agriculture: Concepts, tools, applications, and opportunities. Int J Agric & Biol Eng, 2018; 11(4): 32-44.
[3] He Y, Jiang H, Fang H, Wang Y, Liu Y F. Research progress of intelligent obstacle detection methods of vehicles and their application on agriculture. Transactions of the CSAE, 2018; 34(9): 21-32. (in Chinese)
[4] Yao L J, Hu D, Yang Z D, Li H B, Qian M B. Depth recovery for unstructured farmland road image using an improved SIFT algorithm. Int J Agric & Biol Eng, 2019; 12(4): 141-147.
[5] Pang S, Morris D, Radha H. CLOCs: Camera-LiDAR object candidates fusion for 3D object detection. In: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Las Vegas: IEEE, 2020; pp.10386-10393. doi: 10.1109/IROS45743.2020.9341791.
[6] Cui Y P, Xu H, Wu J Q, Sun Y, Zhao J X. Automatic vehicle tracking with roadside LiDAR data for the connected-vehicles system. IEEE Intelligent Systems, 2019; 34(3): 44-51.
[7] Lyu Y C, Bai L, Huang X M. Real-time road segmentation using lidar data processing on an FPGA. In: 2018 IEEE International Symposium on Circuits and Systems (ISCAS), IEEE, 2018; pp.1-5.
[8] Kisner H, Thomas U. Segmentation of 3D point clouds using a new spectral clustering algorithm without a-priori knowledge. In: VISIGRAPP 2018, 2018; pp.315-322.
[9] Zhang W. Lidar-based road and road-edge detection. In: 2010 IEEE Intelligent Vehicles Symposium. La Jolla: IEEE, 2010; pp.845-848. doi: 10.1109/IVS.2010.5548134.
[10] Charles R Q, Su H, Mo K C, Guibas L J. Pointnet: Deep learning on point sets for 3D classification and segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2017; pp.652-660.
[11] Beltrán J, Guindel C, Moreno F M, Cruzado D, Garcia F, De La Escalera A. Birdnet: A 3D object detection framework from lidar information. In: 2018 21st International Conference on Intelligent Transportation Systems (ITSC), IEEE, 2018; pp.3517-3523.
[12] Hua B S, Tran M K, Yeung S K. Pointwise convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2018; pp.984-993.
[13] Zhang Y H, Wang J, Wang X N, Dolan J M. Road-segmentation-based curb detection method for self-driving via a 3D-LiDAR sensor. IEEE Transactions on Intelligent Transportation Systems, 2018; 19(12): 3981-3991.
[14] Zhou Y, Tuzel O. Voxelnet: End-to-end learning for point cloud based 3D object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, IEEE, 2018; pp.4490-4499.
[15] Chang Y C, Xue F, Sheng F, Liang W T, Ming A L. Fast road segmentation via uncertainty-aware symmetric network. In: 2022 International Conference on Robotics and Automation (ICRA), Philadelphia: IEEE, 2022; pp.1124-11130. doi: 1109/ICRA46639.2022.9812452.
[16] Zhu Z, Liu J L. Graph-based ground segmentation of 3D LIDAR in rough area. In: 2014 IEEE International Conference on Technologies for Practical Robot Applications (TePRA), IEEE, 2014; pp.1–5. doi: 10.1109/TePRA.2014.6869157.
[17] Cheng Z Y, Ren G Q, Zhang Y. Ground segmentation from 3D point cloud using features of scanning line segments. Opto-Electronic Engineering, 2019; 46(7): 180268. doi: 10.12086/OEE.2019.180268.
[18] Triess L T, Peter D, Rist C B, Enzweiler M, Zollner J M. CNN-based synthesis of realistic high-resolution LiDAR data. In: 2019 IEEE Intelligent Vehicles Symposium (IV), Paris: IEEE, 2019; pp.1512-1519.
[19] Shan T X, Wang J K, Chen F F, Szenher P, Englot B Simulation-based lidar super-resolution for ground vehicles. Robotics and Autonomous Systems, 2020 134: 103647. doi: 10.1016/J.ROBOT.2020.103647.
[20] Zhou Z, Siddiquee M M R, Tajbakhsh N, Liang J M. Unet++: Redesigning skip connections to exploit multiscale features in image segmentation. IEEE Transactions on Medical Imaging, 2019; 39(6): 1856-1867.
[21] Ding G G, Han J G, Ding X H, Guo Y C. ACNet: Strengthening the kernel skeletons for powerful CNN via asymmetric convolution blocks. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, IEEE, 2019; pp.1911-1920.
[22] Oktay O, Schlemper J, Folgoc L L, Lee M, Heinrich M, Misawa K, et al. Attention U-Net: Learning where to look for the pancreas. arXiv preprint, 2018; arXiv:1804.03999, 2018.
[23] Yang J D, Zhu J T, Wang H L, Yang X. Dilated MultiResUNet: Dilated multiresidual blocks network based on U-Net for biomedical image segmentation. Biomedical Signal Processing and Control, 2021; 68: 102643. doi: 10.1016/j.bspc.2021.102643.
[24] Bala S A, Kant S. Dense dilated inception network for medical image segmentation. International Journal of Advanced Computer Science and Applications (IJACSA), 2020; 11(11): 0111195. doi: 10.14569/IJACSA.2020.0111195.

Downloads

Published

2023-05-12

How to Cite

Yang, L., Li, Y., Chang, M., Xu, Y., Hu, B., Wang, X., & Wu, C. (2023). Recognition of field roads based on improved U-Net++ Network. International Journal of Agricultural and Biological Engineering, 16(2), 171–178. Retrieved from https://ijabe.migration.pkpps03.publicknowledgeproject.org/index.php/ijabe/article/view/7941

Issue

Vol. 16 No. 2 (2023): IJABE

Section

Information Technology, Sensors and Control Systems

License

IJABE is an international peer reviewed open access journal, adopting Creative Commons Copyright Notices as follows.


Authors who publish with this journal agree to the following terms:


  1. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.

  2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.

  3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).