Method for the automatic recognition of cropland headland images based on deep learning
Keywords:
cropland image, deep learning, image recognition, model compression, MobileNetV2 networkAbstract
For self-driving agricultural vehicles, the sensing of the headland environment based on image recognition is an important technological aspect. Cropland headland environments are complex and diverse. Traditional image feature extraction methods have many limitations. This study proposed a method of automatic cropland headland image recognition based on deep learning. Based on the characteristics of cropland headland environments and practical application needs, a dataset was constructed containing six categories of annotated cropland headland images and an augmented headland image training set was used to train the compact network MobileNetV2. Under the same experimental conditions, the model prediction accuracy for the first ranked category in all the results (Top-1 accuracy) of the MobileNetV2 network on the validation set was 98.5%. Compared with classic ResNetV2-50, Inception-V3, and backend-compressed Inception-V3, MobileNetV2 has a high accuracy, high recognition speed, and a small memory footprint. To further test the performance of the model, 250 images were used for each of the six categories of headland images as the test set for the experiments. The average of the harmonic mean of precision and recall (F1-score) of the MobileNetV2 network for all the categories of headland images reached 97%. The MobileNetV2 network exhibits good robustness and stability. The results of this study indicate that onboard computers on self-driving agricultural vehicles are able to employ the MobileNetV2 network for headland image recognition to meet the application requirements of headland environment sensing. Keywords: cropland image, deep learning, image recognition, model compression, MobileNetV2 network DOI: 10.25165/j.ijabe.20231602.6195 Citation: Qiao Y J, Liu H, Meng Z J, Chen J P, Ma L Y. Method for the automatic recognition of cropland headland images based on deep learning. Int J Agric & Biol Eng, 2023; 16(2): 216-224.References
[1] Hu J T, Gao L, Bai X P, Li T C, Liu X G. Review of research on automatic guidance of agricultural vehicles. Transactions of the CSAE, 2015; 31(10): 1-10. (in Chinese)
[2] Csornai G, László I, Suba Z, Nádor G, Bognár E, Hubik I, et al. 2007. The integrated utilization of satellite images in Hungary: Operational applications from crop monitoring to ragweed control. In: New Developments and Challenges in Remote Sensing, 2007; pp.15-23.
[3] Chen J, Chen T Q, Mei X M, Shao Q F, Deng M. Hilly farmland extraction from high resolution remote sensing imagery based on optimal scale selection. Transactions of the CSAE, 2014; 30(5): 99-107. (in Chinese)
[4] Bay H, Tyutelaars T, Van Gool L. SURF: Speeded up robust features. In: Computer Vision - ECCV 2006, Springer, 2006; pp.404-417. doi: 10.1007/11744023_32.
[5] Lowe D G. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 2004, 60(2):91-110.
[6] Watanabe T, Ito S, Yokoi K. Co-occurrence histograms of oriented gradients for human detection. IPSJ Transactions on Computer Vision and Applications, 2010; 2: 39-47.
[7] Lecun Y, Bengio Y, Haffner P. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 1998; 86(11): 2278-2324.
[8] Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks. Communications of the ACM, 2012; 60(6): 84-90.
[9] Szegedy C, Liu W, Jia Y Q, Sermanet P, Reed S, Anguelov D, et al. Going deeper with convolutions. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, 2015; pp.1-9.
[10] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv, 2014; arXiv: 1409.1556.
[11] He K M, Zhang X Y, Ren S Q, Sun J. Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision & Pattern Recognition (CVPR), IEEE, 2016; pp.770-778.
[12] Zhu S P, Zhu J X, Huang H, Li G L. Wheat Grain Integrity Image Detection System Based on CNN. Transactions of the CSAM, 2020; 51(5): 36-42. (in Chinese)
[13] Yang K, Liu H, Wang P, Meng Z J, Chen J P. Convolutional neural network-based automatic image recognition for agricultural machinery. Int J Agric & Biol Eng, 2018; 11(4): 200-206.
[14] Zhao L X, Hou F D, Lu Z C, Zhu H C, Ding X L. Image recognition of cotton leaf diseases and pests based on transfer learning. Transactions of the CSAE, 2020; 36(7): 184-191. (in Chinese)
[15] Xu J H, Shao M Y, Wang Y C, Han W T. Recognition of corn leaf spot and rust based on transfer learning with convolutional neural network. Transactions of the CSAM, 2020; 51(2): 230-236, 253. (in Chinese)
[16] Kim W-S, Lee D-H, Kim T, Kim G, Kim H, Sim T, et al. One-shot classification-based tilled soil region segmentation for boundary guidance in autonomous tillage. Computers and Electronics in Agriculture, 2021; 189: 106371. doi: 10.1016/j.compag.2021.106371.
[17] He Y, Zhang X Y, Zhang Z Q, Fang H. Automated detection of boundary line in paddy field using MobileV2-UNet and RANSAC. Computers and Electronics in Agriculture, 2022; 194: 106697. doi: 10.1016/j.compag.2022.106697.
[18] Qiao Y J, Yang P S, Meng Z J, Wang Q, Liu H. Detection system of headland boundary line based on machine vision. Journal of Agricultural Mechanization Research, 2022; 44(11): 24-30. (in Chinese)
[19] GB/T 21010-2017. Current land use classification. Ministry of Natural Resources, China, 2017. (in Chinese)
[20] Addo K A. Urban and peri-urban agriculture in developing countries studied using remote sensing and in situ methods. Remote Sensing, 2010; 2(2): 479-513.
[21] Dwork C, Feldman V, Hardt M, Pitassi T, Reingold O, Roth A. The reusable holdout: Preserving validity in adaptive data analysis. Science, 2015; 349(6248): 636-638.
[22] Hinton G, Vinyals O, Dean J. Distilling the knowledge in a neural network. arXiv preprint, 2015; arXiv:1503.02531.
[23] Jaderberg M, Vedaldi A, Zisserman A. Speeding up convolutional neural networks with low rank expansions. arXiv preprint, 2014; arXiv:1405.3866.
[24] Li H, Kadav A, Durdanovic I, Samet H, Graf H P. Pruning filters for efficient convnets. arXiv preprint, 2016; arXiv:1608.08710.
[25] Cai Z W, He X D, Sun J, Vasconcelos N. Deep learning with low precision by half-wave Gaussian quantization. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017; pp.5406-5414. doi: 10.1109/CVPR.2017.574.
[26] Weiss K, Khoshgoftaar T M, Wang D D. A survey of transfer learning. Journal of Big Data, 2016; 3: 9. doi: 10.1186/s40537-016-0043-6.
[27] Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, et al. ImageNet large scale visual recognition challenge. International Journal of Computer Vision, 2015; 115(3): 211-252.
[28] Sandler M, Howard A, Zhu M L, Zhmoginov A. Mobilenetv2: Inverted residuals and linear bottlenecks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018; pp.4510-4520. doi:10.1109/CVPR.2018.00474.
[29] Howard A G, Zhu M, Chen B, Kalenichenko D, Wang W J, Weyand T, et al. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint, 2017; arXiv:1704.04861.
[30] Glorot X, Bordes A, Bengio Y. Deep sparse rectifier neural networks. Journal of Machine Learning Research, 2011; 15: 315-323.
[31] Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R. Dropout: a simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, 2014; 15(1): 1929-1958.
[32] Şimşekli U, Zhu L, Teh Y W, Gürbüzbalaban M. Fractional underdamped langevin dynamics: Retargeting SGD with momentum under heavy-tailed gradient noise. arXiv preprint, 2020; arXiv:2002.05685.
[33] He K M, Zhang X Y, Ren S Q, Sun J. Identity mappings in deep residual networks. In: Computer Vision – ECCV 2016, Springer, 2016; pp.630-645.
[34] Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z. Rethinking the inception architecture for computer vision. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, 2016; pp.2818-2826.
[2] Csornai G, László I, Suba Z, Nádor G, Bognár E, Hubik I, et al. 2007. The integrated utilization of satellite images in Hungary: Operational applications from crop monitoring to ragweed control. In: New Developments and Challenges in Remote Sensing, 2007; pp.15-23.
[3] Chen J, Chen T Q, Mei X M, Shao Q F, Deng M. Hilly farmland extraction from high resolution remote sensing imagery based on optimal scale selection. Transactions of the CSAE, 2014; 30(5): 99-107. (in Chinese)
[4] Bay H, Tyutelaars T, Van Gool L. SURF: Speeded up robust features. In: Computer Vision - ECCV 2006, Springer, 2006; pp.404-417. doi: 10.1007/11744023_32.
[5] Lowe D G. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 2004, 60(2):91-110.
[6] Watanabe T, Ito S, Yokoi K. Co-occurrence histograms of oriented gradients for human detection. IPSJ Transactions on Computer Vision and Applications, 2010; 2: 39-47.
[7] Lecun Y, Bengio Y, Haffner P. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 1998; 86(11): 2278-2324.
[8] Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks. Communications of the ACM, 2012; 60(6): 84-90.
[9] Szegedy C, Liu W, Jia Y Q, Sermanet P, Reed S, Anguelov D, et al. Going deeper with convolutions. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, 2015; pp.1-9.
[10] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv, 2014; arXiv: 1409.1556.
[11] He K M, Zhang X Y, Ren S Q, Sun J. Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision & Pattern Recognition (CVPR), IEEE, 2016; pp.770-778.
[12] Zhu S P, Zhu J X, Huang H, Li G L. Wheat Grain Integrity Image Detection System Based on CNN. Transactions of the CSAM, 2020; 51(5): 36-42. (in Chinese)
[13] Yang K, Liu H, Wang P, Meng Z J, Chen J P. Convolutional neural network-based automatic image recognition for agricultural machinery. Int J Agric & Biol Eng, 2018; 11(4): 200-206.
[14] Zhao L X, Hou F D, Lu Z C, Zhu H C, Ding X L. Image recognition of cotton leaf diseases and pests based on transfer learning. Transactions of the CSAE, 2020; 36(7): 184-191. (in Chinese)
[15] Xu J H, Shao M Y, Wang Y C, Han W T. Recognition of corn leaf spot and rust based on transfer learning with convolutional neural network. Transactions of the CSAM, 2020; 51(2): 230-236, 253. (in Chinese)
[16] Kim W-S, Lee D-H, Kim T, Kim G, Kim H, Sim T, et al. One-shot classification-based tilled soil region segmentation for boundary guidance in autonomous tillage. Computers and Electronics in Agriculture, 2021; 189: 106371. doi: 10.1016/j.compag.2021.106371.
[17] He Y, Zhang X Y, Zhang Z Q, Fang H. Automated detection of boundary line in paddy field using MobileV2-UNet and RANSAC. Computers and Electronics in Agriculture, 2022; 194: 106697. doi: 10.1016/j.compag.2022.106697.
[18] Qiao Y J, Yang P S, Meng Z J, Wang Q, Liu H. Detection system of headland boundary line based on machine vision. Journal of Agricultural Mechanization Research, 2022; 44(11): 24-30. (in Chinese)
[19] GB/T 21010-2017. Current land use classification. Ministry of Natural Resources, China, 2017. (in Chinese)
[20] Addo K A. Urban and peri-urban agriculture in developing countries studied using remote sensing and in situ methods. Remote Sensing, 2010; 2(2): 479-513.
[21] Dwork C, Feldman V, Hardt M, Pitassi T, Reingold O, Roth A. The reusable holdout: Preserving validity in adaptive data analysis. Science, 2015; 349(6248): 636-638.
[22] Hinton G, Vinyals O, Dean J. Distilling the knowledge in a neural network. arXiv preprint, 2015; arXiv:1503.02531.
[23] Jaderberg M, Vedaldi A, Zisserman A. Speeding up convolutional neural networks with low rank expansions. arXiv preprint, 2014; arXiv:1405.3866.
[24] Li H, Kadav A, Durdanovic I, Samet H, Graf H P. Pruning filters for efficient convnets. arXiv preprint, 2016; arXiv:1608.08710.
[25] Cai Z W, He X D, Sun J, Vasconcelos N. Deep learning with low precision by half-wave Gaussian quantization. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017; pp.5406-5414. doi: 10.1109/CVPR.2017.574.
[26] Weiss K, Khoshgoftaar T M, Wang D D. A survey of transfer learning. Journal of Big Data, 2016; 3: 9. doi: 10.1186/s40537-016-0043-6.
[27] Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, et al. ImageNet large scale visual recognition challenge. International Journal of Computer Vision, 2015; 115(3): 211-252.
[28] Sandler M, Howard A, Zhu M L, Zhmoginov A. Mobilenetv2: Inverted residuals and linear bottlenecks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018; pp.4510-4520. doi:10.1109/CVPR.2018.00474.
[29] Howard A G, Zhu M, Chen B, Kalenichenko D, Wang W J, Weyand T, et al. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint, 2017; arXiv:1704.04861.
[30] Glorot X, Bordes A, Bengio Y. Deep sparse rectifier neural networks. Journal of Machine Learning Research, 2011; 15: 315-323.
[31] Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R. Dropout: a simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, 2014; 15(1): 1929-1958.
[32] Şimşekli U, Zhu L, Teh Y W, Gürbüzbalaban M. Fractional underdamped langevin dynamics: Retargeting SGD with momentum under heavy-tailed gradient noise. arXiv preprint, 2020; arXiv:2002.05685.
[33] He K M, Zhang X Y, Ren S Q, Sun J. Identity mappings in deep residual networks. In: Computer Vision – ECCV 2016, Springer, 2016; pp.630-645.
[34] Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z. Rethinking the inception architecture for computer vision. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, 2016; pp.2818-2826.
Downloads
Published
2023-05-12
How to Cite
Qiao, Y., Liu, H., Meng, Z., Chen, J., & Ma, L. (2023). Method for the automatic recognition of cropland headland images based on deep learning. International Journal of Agricultural and Biological Engineering, 16(2), 216–224. Retrieved from https://ijabe.migration.pkpps03.publicknowledgeproject.org/index.php/ijabe/article/view/6195
Issue
Section
Information Technology, Sensors and Control Systems
License
IJABE is an international peer reviewed open access journal, adopting Creative Commons Copyright Notices as follows.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
