Nondestructive perception of potato quality in actual online production based on cross-modal technology

Authors

  • Qiquan Wei 1. School of Automation, Hangzhou Dianzi University, Hangzhou 310018, China
  • Yurui Zheng 1. School of Automation, Hangzhou Dianzi University, Hangzhou 310018, China
  • Zhaoqing Chen 2. School of Information Engineering and Internet of Things, Huzhou Vocational and Technical College, Huzhou 313000, Zhejiang, China
  • Yun Huang 3. Jinhua Academy of Agricultural Sciences, Jinhua 321017, Zhejiang, China
  • Changqing Chen 3. Jinhua Academy of Agricultural Sciences, Jinhua 321017, Zhejiang, China
  • Zhenbo Wei 4. College of Biosystems Engineering and Food Science, Zhejiang University, Hangzhou 310058, China
  • Shuiqin Zhou 5. Fair Friend Institute of Intelligent Manufacturing, Hangzhou Vocational and Technical College, Hangzhou 310018, China
  • Hongwei Sun 1. School of Automation, Hangzhou Dianzi University, Hangzhou 310018, China
  • Fengnong Chen 1. School of Automation, Hangzhou Dianzi University, Hangzhou 310018, China

Keywords:

cross-modal technology, potato quality, YOLOv5s, VIS/NIR spectroscopy, online nondestructive detection

Abstract

Nowadays, China stands as the global leader in terms of potato planting area and total potato production. The rapid and nondestructive detection of the potato quality before processing is of great significance in promoting rural revitalization and augmenting farmers’ income. However, existing potato quality sorting methods are primarily confined to theoretical research, and the market lacks an integrated intelligent detection system. Therefore, there is an urgent need for a post-harvest potato detection method adapted to the actual production needs. The study proposes a potato quality sorting method based on cross-modal technology. First, an industrial camera obtains image information for external quality detection. A model using the YOLOv5s algorithm to detect external green-skinned, germinated, rot and mechanical damage defects. VIS/NIR spectroscopy is used to obtain spectral information for internal quality detection. A convolutional neural network (CNN) algorithm is used to detect internal blackheart disease defects. The mean average precision (mAP) of the external detection model is 0.892 when intersection of union (IoU) = 0.5. The accuracy of the internal detection model is 98.2%. The real-time dynamic defect detection rate for the final online detection system is 91.3%, and the average detection time is 350 ms per potato. In contrast to samples collected in an ideal laboratory setting for analysis, the dynamic detection results of this study are more applicable based on a real-time online working environment. It also provides a valuable reference for the subsequent online quality testing of similar agricultural products. Keywords: cross-modal technology, potato quality, YOLOv5s, VIS/NIR spectroscopy, online nondestructive detection DOI: 10.25165/j.ijabe.20231606.8076 Citation: Wei Q Q, Zheng Y R, Chen Z Q, Huang Y, Chen C Q, Wei Z B, et al. Nondestructive perception of potato quality in actual online production based on cross-modal technology. Int J Agric & Biol Eng, 2023; 16(6): 280-290.

References

[1] Xie C H. Potato Industry: Status and Development. Journal of Huazhong Agricultural University (Social Sciences Edition), 2012; 97(1): 1–4. URL: http://hnxbl.cnjournals.net/hznydxsk/article/abstract/20120101?st=article_issue
[2] Jing J, Li J W, Liao G P, Yu X J, Viray C. Methodology for Potatoes Defects Detection with Computer Vision. Proceedings. The 2009 International Symposium on Information Processing (ISIP 2009), 2009; 346. URL: https://www.researchgate.net/publication/242098282_Methodology_for_Potatoes_Defects_Detection_with_Computer_Vision
[3] Ebrahimi E, Mollazade K, Arefi A. Detection of Greening in Potatoes using Image Processing Techniques. Journal of American Science, 2011; 7(3): 243–247. URL: https://www.researchgate.net/publication/243457611_Detection_of_Greening_in_Potatoes_using_Image_Processing_Techniques
[4] Barnes M, Duckett T, Cielniak G, Stroud G, Harper G. Visual detection of blemishes in potatoes using minimalist boosted classifiers. Journal of Food Engineering, 2010; 98(3): 339–346. URL: https://www.mendeley.com/catalogue/
55596dd8-9d7c-3992-8fdf-2d9ce49f2853/
[5] Elmasry G, Cubero S, Moltó E, Blasco J. l. In-line sorting of irregular potatoes by using automated computer-based machine vision system. Journal of Food Engineering, 2012; 112(1-2): 60-68. URL: https://www.sciencedirect.com/science/
article/abs/pii/S0260877412001690
[6] Oppenheim D, Shani G. Potato Disease Classification Using Convolution Neural Networks. Advances in Animal Biosciences, 2017; 8(02): 244-249. URL: https://www.sciencedirect.com/science/article/abs/pii/S2040470017001376
[7] Elsharif A A, Dheir I M, Mettleq A, Abu-Naser S S. Potato Classification Using Deep Learning. International Journal of Academic Pedagogical Research, 2020; 3(12): 1-8. URL: https://philpapers.org/rec/ELSPCU
[8] Chen J D, Zhang D F, Nanehkaran Y A, Li D L. Detection of rice plant diseases based on deep transfer learning. Journal of the Science of Food and Agriculture, 2020; 100(7): 3246-3256. URL: https://onlinelibrary.wiley.com/doi/abs/10.1002/
jsfa.10365
[9] Zhao G Y, Quan L X, Li H L, Feng H Q, Li S W, Zhang S H, et al. Real-time recognition system of soybean seed full-surface defects based on deep learning. Computers and Electronics in Agriculture, 2021; 187:106230. URL: https://www.sciencedirect.com/science/article/abs/pii/S0168169921002477
[10] Ramos R P, Gomes J S, Prates R M, Filho E F, Teruel B J, Costa D S. Non‐invasive setup for grape maturation classification using deep learning. Journal of the Science of Food and Agriculture, 2021; 101(5): 2042-2051. URL: https://onlinelibrary.wiley.com/doi/abs/10.1002/jsfa.10824
[11] Thuyet D Q, Kobayashi Y C, Matsuo M. A robot system equipped with deep convolutional neural network for autonomous grading and sorting of root-trimmed garlics. Computers and Electronics in Agriculture, 2020; 178:105727. URL: https://www.sciencedirect.com/science/article/abs/pii/S0168169920310140
[12] Nouri-Ahmadabadi H, Omid M, Mohtasebi S S, Firouz M S. Design, Development and Evaluation of an Online Grading System for Peeled Pistachios Equipped with Machine Vision Technology and Support Vector Machine. Information Processing in Agriculture, 2017; 4(4): 333–341. URL: https://www.sciencedirect.com/
science/article/pii/S2214317316300117
[13] Blasco J, Cubero S, Gómez-Sanchís J, Mira P, Moltó E. Development of a machine for the automatic sorting of pomegranate (Punica granatum) arils based on computer vision. Journal of Food Engineering, 2009; 90(1): 27-34. URL: https://www.sciencedirect.com/science/article/abs/pii/S0260877408002653
[14] Hajjar G, Quellec S, Pépin J, Challois S, Joly G, Deleu C, et al. MRI investigation of internal defects in potato tubers with particular attention to rust spots induced by water stress. Postharvest Biology and Technology, 2021; 180:111600. URL: https://www.sciencedirect.com/science/article/abs/pii/S0925521421001393
[15] Sosa P , Guild G , Burgos G , Bonierbale M, Felde T. Potential and application of X-ray fluorescence spectrometry to estimate iron and zinc concentration in potato tubers. Journal of Food Composition and Analysis, 2018; 70: 22-27. URL: https://www.sciencedirect.com/science/article/pii/S088915751830070X
[16] López-Maestresalas A, Keresztes J C , Goodarzi M , Arazuri S, Jarén C, Saeys W. Non-destructive detection of blackspot in potatoes by Vis-NIR and SWIR hyperspectral imaging. Food Control, 2016; 70: 229-241. URL: https://www.sciencedirect.com/science/article/abs/pii/S0956713516303085
[17] Wu L G, He J G, Liu G S, Wang S L, He X G. Detection of common defects on jujube using Vis-NIR and NIR hyperspectral imaging. Postharvest Biology and Technology, 2016; 112: 134-142. URL: https://www.sciencedirect.com/science/
article/abs/pii/S0925521415301083
[18] Ye D D, Sun L J, Tan W Y, Che W K, Yang M C. Detecting and classifying minor bruised potato based on hyperspectral imaging. Chemometrics and Intelligent Laboratory Systems, 2018: 177: 129-139. URL: https://www.sciencedirect.com/
science/article/abs/pii/S0169743917306251
[19] Jamshidi B , Minaei S , Mohajerani E , Ghassemian H. Reflectance Vis/NIR spectroscopy for nondestructive taste characterization of Valencia oranges. Computers and Electronics in Agriculture, 2012; 85: 64-69. URL: https://www.
sciencedirect.com/science/article/abs/pii/S0168169912000828
[20] Peirs A, Lammertyn J, Ooms K, Nicolaı̈ B M. Prediction of the optimal picking date of different apple cultivars by means of VIS/NIR-spectroscopy. Postharvest Biology and Technology, 2001; 21(2): 189-199. URL: https://www.sciencedirect.
com/science/article/abs/pii/S0925521400001459
[21] Tušek A J, Benković M, Malešić E, Marić L, Jurina T, Kljusurić J G. Rapid quantification of dissolved solids and bioactives in dried root vegetable extracts using near infrared spectroscopy. Spectrochimica acta Part A: Molecular and biomolecular spectroscopy, 2021; 261: 120074. URL: https://www.sciencedirect.
com/science/article/abs/pii/S138614252100651X
[22] Moomkesh S , Mireei S A , Sadeghi M , Nazeri M. Early detection of freezing damage in sweet lemons using Vis/SWNIR spectroscopy. Biosystems Engineering, 2017; 164: 157-170. URL: https://www.sciencedirect.com/science/article/abs/
pii/S1537511017305007
[23] Li J B, Huang W Q, Zhao C J, Zhang B H. A comparative study for the quantitative determination of soluble solids content, pH and firmness of pears by Vis/NIR spectroscopy. Journal of food engineering, 2013; 116(2): 324-332.
[24] Scalisi A , O'Connell M G. Application of Visible/NIR spectroscopy for the estimation of soluble solids, dry matter and flesh firmness in stone fruits. Journal of the Science of Food and Agriculture, 2021; 101(5):2100-2107. URL: https://www.sciencedirect.com/science/article/abs/pii/S0260877412005596
[25] Wang F, Li Y Y, Peng Y , Yang B N, Li L, Yin X Q. Hand-held Device for Non-destructive Detection of Potato Quality Parameters. Transactions of the Chinese Society for Agricultural Machinery, 2018; (49)7: 7. URL: https://kns.cnki.net/kcms/detail/detail.aspx?dbcode=CJFD&dbname=CJFDLAST2018&filename=NYJX201807042&uniplatform=NZKPT&v=Bhl832oeBgzPg99FnP-kZUbLtMML3R4SrSrF4CgEMvMDHTIFsyL_OYfdCcMM3L7W
[26] Wang F, Li Y Y, Peng Y K, Yang B N, Li L, Liu Y C. Multi-Parameter Potato Quality Non-Destructive Rapid Detection by Visible/Near-Infrared Spectra. Spectroscopy and Spectral Analysis, 2018; 38(12): 3736. URL: https://kns.cnki.net/kcms/detail/detail.aspx?dbcode=CJFD&dbname=CJFDLAST2019&filename=GUAN201812016&uniplatform=NZKPT&v=yiHXcRaOJAICFkcD9KtCct5Emf1NoONfuDjHKdAkR5LfKG57CS5m0Iwfdl3jFTBI
[27] Zhang X Y , Liu W , Xing L , et al. An Near-infrared Prediction Model for Quality Indexes of Potato Processing. infrared, 2012.
[28] Zhu Z, Zeng S W, Li X Y, Zheng J. Nondestructive Detection of Blackheart in Potato by Visible/Near Infrared Transmittance Spectroscopy. Journal of Spectroscopy, 2015; 2015: 1-9. URL: https://kns.cnki.net/kcms/detail/
detail.aspx?dbcode=CJFD&dbname=CJFD2012&filename=HWAI201212009&uniplatform=NZKPT&v=P0yTJMNtpDOrTITF6rfUe2KOEie_0PcVOwdUhPUMDxa1w5nlgHpOsWlOYznyIAFm
[29] Song Y, Wang X Z, Xie H L, Li L Q, Ning J M, Zhang Z Z. Quality Evaluation of Keemun black tea by fusing data obtained from near-infrared reflectance spectroscopy and computer vision sensors. Spectrochimica Acta Part A: Molecular and Biomolecular Spectroscopy, 2021; 252(5): 119522. URL: https://www.sciencedirect.com/science/article/abs/pii/S1386142521000986
[30] Yu H L, Liang Y L, Liang H, Zhang Y Z. Recognition of wood surface defects with near infrared spectroscopy and machine vision. Journal of Forestry Research, 2019; 30(6): 2379-2386. URL: https://link.springer.com/article/10.1007/s11676-018-00874-w
[31] Huang X Y, Xu H X, Wu L, Dai H, Yao L Y, Han F K. A data fusion detection method for fish freshness based on computer vision and near-infrared spectroscopy. Analytical Methods, 2016; 8(14): 2929-2935. URL: https://pubs.rsc.org/en/
content/articlelanding/2016/ay/c5ay03005f/unauth
[32] Yin J F, Hameed S, Xie L J, Ying Y B. Non-destructive detection of foreign contaminants in toast bread with near infrared spectroscopy and computer vision techniques. Journal of Food Measurement and Characterization, 2021; 15(1): 189–198. URL: https://link.springer.com/article/10.1007/s11694-020-00627-6
[33] Ministry of Agriculture of the PRC. Grades and specifications of potatoes, 2006.
[34] Yafen Han, Chengxu Lu¨, Yanwei Yuan, Bingnan Yang, Zhao Qingliang, Cao Youfu, and Yin Xueqing. Pls-discriminant analysis on patato blackheart disease based on vis-nir transmission spectroscopy. Spectroscopy and Spectral Analysis, 41(4):1213, 2021. URL: https://kns.cnki.net/kcms/detail/detail.aspx?dbcode=CJFD&dbname=
CJFDLAST2021&filename=GUAN202104044&uniplatform=NZKPT&v=w9Z0y1AQjjAudCYEmiOnS8srLZdbyPiVv5HBjrn9LPTri15Zm5X57fHem8eJp4pZ
[35] Jubayer F, Soeb J A, Mojumder A N, Paul M K, Barua P, Kayshar S, Akter S S, et al. Detection of mold on the food surface using yolov5. Current Research in Food Science, 2021; 4:724–728. URL: https://www.sciencedirect.com/
science/article/pii/S2665927121000812
[36] Rong D, Wang H Y, Ying Y B, Zhang Z Y, Zhang Y S. Peach variety detection using VIS-NIR spectroscopy and deep learning. Computers and Electronics in Agriculture, 2020; 175: 105553. URL: https://www.sciencedirect.com/
science/article/abs/pii/S0168169920305354
[37] Tazehkandi A A. Computer Vision with OpenCV 3 and Qt5: Build visually appealing, multithreaded, cross-platform computer vision applications. Packt Publishing Ltd, 2018.
[38] Yao L J, Lu L, Zheng R. Study on detection method of external defects of potato image in visible light environment. 2017 10th International Conference on Intelligent Computation Technology and Automation (ICICTA), 2017; 118–122. URL: https://www.researchgate.net/publication/320829726_Study_on_
Detection_Method_of_External_Defects_of_Potato_Image_in_Visible_Light_Environment
[39] Clark C J, McGlone V A, Jordan R B. Detection of Brownheart in ‘Braeburn’ apple by transmission NIR spectroscopy. Postharvest Biology and Technology, 2003: 28(1): 87-96. URL: https://www.sciencedirect.com/science/
article/abs/pii/S0925521402001229

Downloads

Published

2024-02-06

How to Cite

Wei, Q., Zheng, Y., Chen, Z., Huang, Y., Chen, C., Wei, Z., … Chen, F. (2024). Nondestructive perception of potato quality in actual online production based on cross-modal technology. International Journal of Agricultural and Biological Engineering, 16(6), 280–290. Retrieved from https://ijabe.migration.pkpps03.publicknowledgeproject.org/index.php/ijabe/article/view/8076

Issue

Section

Agro-product and Food Processing Systems