ERI: Object Recognition Beyond Color and Shape

About This Grant

Machine vision applications have become increasingly important and are becoming an integral part of our everyday life. This is fueled by the rapid increase of automation in industrial settings. While the demand is high, machine vision still faces significant challenges and is unable to complete vision tasks that are easier for humans. Such dilemma is primarily attributed to the limitations of shape and color-based object recognition paradigm that is being currently used. This research effort is aimed at developing an innovative solution to realize computer vision beyond color and shape to enable extensive application in manufacturing, agriculture, cybersecurity, transportation, construction, and biomedicine. The proposed laser object recognition method will enable advancements and understanding of object recognition via transformative computer vision methods. The research will lead to significant impact on object recognition in dark and cluttered environments, which is known to be difficult using conventional computer vision methods. This project proposes a novel object recognition method for confident object recognition. In contrast to existing shape and color-based object recognition methods, where limited visual information is utilized, the new laser object recognition method is designed to rely upon harnessing the laser-material interaction information to conduct object recognition. The proposed laser object recognition method is fundamentally rooted in the laws of physics for laser reflection and scattering from materials. The proposed method projects laser dots onto the objects of interest. From the detection of the reflected light and the scattering patterns, object recognition is carried out. Object recognition data reduction pipelines will be developed to process the data. The proposed research consists of two thrusts. First, hardware will be built for data collection and object recognition algorithms will be developed. Second, series of object recognition robustness tests and optimization will be conducted to validate the performance of the method. The proposed laser object recognition method will be able to achieve what has not been feasible using conventional shape and color-based object recognition methods. The proposed methodology will be applied to material recognition, close proximity object recognition and same color and shape object differentiation, among other possibilities. The research will have wide range of educational outcomes by recruiting students for the full research cycle from hardware and software platform construction to the optimization of the accuracy of the laser object recognition method. It is expected that the project will inspire students’ interests to pursue smart sensing and machine vision-related research and professional careers to benefit STEM education and prepare the future skilled workforce. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.