CPS: Medium: An Autonomous Robotic System for Precision and High-Throughput Tomato Phenotyping in Large-Scale Greenhouses

About This Grant

The goal of this project is to develop an autonomous robotic system for precision and high-throughput tomato phenotyping in large-scale greenhouses. This system consists of computer vision-based phenotyping with a mobile robot arm that can access the plant canopy for high-quality image acquisition and a dynamic wireless charger that can provide an uninterruptible power source for the entire system. This proposed system is exceptionally distinctive, unique, and advanced compared to the existing autonomous phenotyping systems. This autonomous robotic phenotyping system provides a noninvasive and non-destructive way of accurately obtaining phenotypic information from individual plants. It is also able to accommodate greenhouse workers’ specific needs, perform high-throughput phenotyping by measuring various traits simultaneously, and provide feedback based on autonomous phenotyping evaluation. Automating tasks by using this developed robotic phenotyping system can significantly improve agricultural workers’ well-being by mitigating work hazards and reducing the duration of time they spend in the greenhouse assessing plant status. Four research goals are proposed: (1) Develop deep learning models to localize the tomato plants, perceive the greenhouse environment, and select the target fruits for phenotyping by integrating domain knowledge in phenotyping and greenhouse managers’ specific needs. (2) Develop novel robot motion planning algorithms to take high-quality images for phenotyping and prevent potential damage to the plants and robot. (3) Develop a multi-task learning model to compute the diverse dozens of tomato fruit traits and automatically evaluate the quality of the phenotyping results based on uncertainty analysis and domain knowledge to determine if phenotyping needs to be redone, which will close the loop of this system. (4) Develop an optimized high-efficiency, high-reliability, and low-cost wireless dynamic battery charger concept to provide power to the autonomous robotic phenotyping system operating uninterruptedly in large-scale and humid greenhouses. The fundamental scientific contributions to advance the knowledge in multiple disciplines are::In computer vision, an effective and energy-saving phenotyping recognition method adapting large AI models on relatively small tomato datasets with multi-modality input data (text, image, depth) will be developed. In robot motion planning, a novel motion planning algorithm will be developed to guarantee high-quality image acquisition, avoid obstacles, and prevent plants from damage. In power electronics, a novel circuit topology and a multi-objective design optimization algorithm will be developed to concurrently achieve high efficiency, high reliability, high power density, and low cost of the dynamic wireless charger. From a broader impact perspective, the research approaches will enable greater farm productivity, provide improved safety and health environment for workers, and can be translated to other application domains. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.