Development of a visuo-tactile sensor for non-destructive peach firmness and contact force measurement suitable for robotic arm applications.

Precise measurement of firmness was crucial for determining optimal harvesting times, implementing rational storage strategies and minimizing avoidable waste. Current technologies for assessing peach firmness struggled to balance high precision and non-destructive methods, while demonstrating high sensitivity to environmental disturbances, thereby limiting their application to production line. Future various scenarios in agriculture would increasingly rely on robotic arms, yet existing firmness assessment technologies were not compatible with these automated systems. Additionally, monitoring contact force was essential for flexible operation of the robotic arms. This work introduced a visuo-tactile sensor equipped with markers to capable of measuring peach firmness and monitoring contact force simultaneously during a single contact process, making it suitable for robotic arm applications. The contact was operated by the texture analyzer to simulate the fruit grasping process by a robotic arm. Utilizing deep neural networks and machine learning-based techniques to process high-precision geometric images collected by an internal camera, the visuo-tactile sensor achieved non-destructive measurements of peach firmness and contact force. For firmness measurement in the test set, the sensor achieved coefficient of determination (R) of 0.878 and a root mean square error (RMSE) of 0.732. For contact force detection, the R was 0.942, and RMSE was 1.115 in the test set. The results showed visuo-tactile sensor was feasible for non-destructive detection of peach firmness and contact force, and has a broad application prospect in the field of agricultural robotics.