Digital twins in smart farming: An autoware-based simulator for autonomous agricultural vehicles

Authors

  • Xin Zhao 1. College of Information and Electrical Engineering, China Agricultural University, Beijing 100083, China; 2. Key Laboratory of Agricultural Machinery Monitoring and Big Data Application, Ministry of Agriculture and Rural Affairs, Beijing 100083, China http://orcid.org/0000-0002-0985-9829
  • Wanli Wang 3. The Bureau of Agriculture and Rural Affairs of Urumchi, Xinjiang Uygur Autonomous Region, Urumchi 830000, China
  • Long Wen 1. College of Information and Electrical Engineering, China Agricultural University, Beijing 100083, China; 2. Key Laboratory of Agricultural Machinery Monitoring and Big Data Application, Ministry of Agriculture and Rural Affairs, Beijing 100083, China
  • Zhibo Chen 1. College of Information and Electrical Engineering, China Agricultural University, Beijing 100083, China; 2. Key Laboratory of Agricultural Machinery Monitoring and Big Data Application, Ministry of Agriculture and Rural Affairs, Beijing 100083, China
  • Sixian Wu 1. College of Information and Electrical Engineering, China Agricultural University, Beijing 100083, China; 2. Key Laboratory of Agricultural Machinery Monitoring and Big Data Application, Ministry of Agriculture and Rural Affairs, Beijing 100083, China
  • Kun Zhou 4. Research & Advanced Engineering, AGCO A/S, DK-8930 Randers, Denmark
  • Mengyao Sun 5. Beijing Agricultural Machinery Experiment Appraisal Extension Station, Beijing 100079, China
  • Lanjun Xu 5. Beijing Agricultural Machinery Experiment Appraisal Extension Station, Beijing 100079, China
  • Bingbing Hu 6. Kunlun Beidou Intelligence Technologies Co. Ltd., Beijing 102200, China
  • Caicong Wu 1. College of Information and Electrical Engineering, China Agricultural University, Beijing 100083, China; 2. Key Laboratory of Agricultural Machinery Monitoring and Big Data Application, Ministry of Agriculture and Rural Affairs, Beijing 100083, China

Keywords:

autoware, simulation platform, autonomous agricultural vehicle, digital twin, autonomous robots

Abstract

Digital twins can improve the level of control over physical entities and help manage complex systems by integrating a range of technologies. The autonomous agricultural machine has shown revolutionary effects on labor reduction and utilization rate in field works. Autonomous vehicles in precision agriculture have the potential to improve competitiveness compared to current crop production methods and have become a research hotspot. However, the development time and resources required in experiments have limited the research in this area. Simulation tools in unmanned farming that are required to enable more efficient, reliable, and safe autonomy are increasingly demanding. Inspired by the recent development of an open-source virtual simulation platform, this study proposed an autoware-based simulator to evaluate the performance of agricultural machine guidance based on digital twins. Oblique photogrammetry using drones is used to construct three-dimensional maps of fields at the same scale as reality. A communication format suitable for agricultural machines was developed for data input and output, along with an inter-node communication methodology. The conversion, publishing, and maintenance of multiple coordinate systems were completed based on ROS (Robot Operating System). Coverage path planning was performed using hybrid curves based on Bézier curves, and it was tested in both a simulation environment and actual fields with the aid of Pure Pursuit algorithms and PID controllers. Keywords: autoware, simulation platform, autonomous agricultural vehicle, digital twin; autonomous robots DOI: 10.25165/j.ijabe.20231604.8039 Citation: Zhao X, Wang W L, Wen L, Chen Z B, Wu S X, Zhou K, et al. Digital twins in smart farming: An autoware-based simulator for autonomous agricultural vehicles. Int J Agric & Biol Eng, 2023; 16(4): 185-190.

References

[1] Bochtis D D, Sørensen C G C, Busato P. Advances in agricultural machinery management: A review. Biosystems Engineering, 2014; 126: 69–81.
[2] Moysiadis V, Tsolakis N, Katikaridis D, Sørensen C G, Pearson S, Bochtis D. Mobile robotics in agricultural operations: A narrative review on planning aspects. Applied Sciences, 2020; 10(10): 3453. doi: 10.3390/app10103453.
[3] Oksanen T, Visala A. Coverage path planning algorithms for agricultural field machines. Journal of Field Robotics, 2009; 26(8): 651–668.
[4] Bochtis D D, Vougioukas S G. Minimising the non-working distance travelled by machines operating in a headland field pattern. Biosystems Engineering, 2008; 101(1): 1–12.
[5] Jensen K, Larsen M, Nielsen S H, Larsen L B, Olsen K S, Jørgensen R N. Towards an open software platform for field robots in precision agriculture. Robotics, 2014; 3(2): 207–234.
[6] Zhang Y H, Wang J, Wang X N, Dolan J M. Road-segmentation-based curb detection method for self-driving via a 3D-LiDAR sensor. IEEE Trans Intell Transp Syst, 2018; 19(12): 3981–3991.
[7] Yao L J, Hu D, Yang Z D, Li H B, Qian M B. Depth recovery for unstructured farmland road image using an improved SIFT algorithm. Int J Agric Biol Eng, 2019; 12(4): 141–147.
[8] Yang L L, Li Y B, Chang M S, Xu Y Y, Hu B B, Wang X X, et al. Recognition of field roads based on improved U-Net plus plus Network. Int J Agric & Biol Eng, 2023; 16(2): 171–178.
[9] Grieves M. Digital twin : Manufacturing excellence through virtual factory replication. Digital Twin White Papaper, 2014; 7p.
[10] Kritzinger W, Karner M, Traar G, Henjes J, Sihn W. Digital Twin in manufacturing: A categorical literature review and classification. IFAC-PapersOnLine, 2018; 51(11): 1016–1022.
[11] Caputo F, Greco A, Fera M, Macchiaroli R. Digital twins to enhance the integration of ergonomics in the workplace design. International Journal of Industrial Ergonomics, 2019; 71: 20–31.
[12] Sivalingam K, Sepulveda M, Spring M, Davies P. A review and methodology development for remaining useful life prediction of offshore fixed and floating wind turbine power converter with digital twin technology perspective. In: 2018 2nd International Conference on Green Energy and Applications, 2018; pp.197–204.
[13] Rong G D, Shin B H, Tabatabaee H, Lu Q, Lemke S, Možeiko M, et al. LGSVL simulator: A high fidelity simulator for autonomous driving. 2020; pp.1-6. doi: 10.1109/ITSC45102.2020.9294422.
[14] Pylianidis C, Osinga S, Athanasiadis I N. Introducing digital twins to agriculture. Computers and Electronics in Agriculture, 2021; 184: 105942. doi: 10.1016/j.compag.2020.105942.
[15] Zhao Y P, Lian L K, Bi C W, Xu Z J. Digital twin for rapid damage detection of a fixed net panel in the sea. Computers and Electronics in Agriculture, 2022; 200: 107247. doi: 10.1016/j.compag.2022.107247.
[16] Tsolakis N, Bechtsis D, Bochtis D. Agros: A robot operating system based emulation tool for agricultural robotics. Agronomy, 2019; 9(7): 403. doi: 10.3390/agronomy9070403.
[17] Dosovitskiy A, Ros G, Codevilla F, López A, Koltun V. CARLA: An open urban driving simulator. In: 1st Conference on Robot Learning (CoRL 2017), PMLR; 2017; 78: 1–16.
[18] Ziegler S, Höpler R. Extending the IPG CarMaker by FMI compliant units. In: Proceedings of the 8th International Modelica Conference, Dresden: Technical Univeristy, Citeseer, 2011; pp.779–783. doi: 10.3384/ecp11063779.
[19] Benekohal R F, Treiterer J. CARSIM: Car-following model for simulation of traffic in normal and stop-and-go conditions. Transportation Research Record Journal of the Transportation Research Board, 1988; 1194: 99–111.
[20] Ryan R R. ADAMS—Multibody system analysis software. In: Multibody Systems Handbook, Springer, 1990; pp.361–402.
[21] Yang Y, Li Y K, Wen X, Zhang G, Ma Q L, Cheng S K, et al. An optimal goal point determination algorithm for automatic navigation of agricultural machinery: Improving the tracking accuracy of the Pure Pursuit algorithm. Computers and Electronics in Agriculture, 2022; 194: 106760. doi: 10.1016/j.compag.2022.106760.
[22] Foote T. tf: The transform library. In: 2013 IEEE Conference Technologies Practical Robot Applications (TePRA), Woburn: IEEE, 2013; pp.1–6. doi: 10.1109/TePRA.2013.6556373.
[23] Hu H M, Kaizu Y, Zhang H D, Xu Y W, Imou K, Li M, et al. Recognition and localization of strawberries from 3D binocular cameras for a strawberry picking robot using coupled YOLO/Mask R-CNN. Int J Agric & Biol Eng, 2022; 15(6): 175–179.
[24] Bao X L, Leng J S, Mao J C, Chen B Y. Cutting of sheep carcass using 3D point cloud with dual-robot system. Int J Agric Biol Eng, 2022; 15(5): 163–171.
[25] Omasa K, Ono E, Ishigami Y, Shimizu Y, Araki Y. Plant functional remote sensing and smart farming applications. Int J Agric & Biol Eng, 2022; 15(4): 1–6.

Downloads

Published

2023-10-17

How to Cite

Zhao, X., Wang, W., Wen, L., Chen, Z., Wu, S., Zhou, K., … Wu, C. (2023). Digital twins in smart farming: An autoware-based simulator for autonomous agricultural vehicles. International Journal of Agricultural and Biological Engineering, 16(4), 184–189. Retrieved from https://ijabe.migration.pkpps03.publicknowledgeproject.org/index.php/ijabe/article/view/8039

Issue

Vol. 16 No. 4 (2023): IJABE

Section

Information Technology, Sensors and Control Systems

License

IJABE is an international peer reviewed open access journal, adopting Creative Commons Copyright Notices as follows.


Authors who publish with this journal agree to the following terms:


  1. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.

  2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.

  3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).