Robust and Dynamics-Aware Active Perception for Space Robotics

About This Grant

Satellites serve as the backbone of our digital world, enabling global communications, navigation systems, weather forecasting, and internet connectivity in remote areas. They facilitate everything from GPS navigation in smartphones to international financial transactions and military operations. When satellite systems fail, the consequences are immediate and far-reaching: navigation systems become unreliable, affecting transportation and logistics; communication networks experience outages, particularly in remote areas dependent on satellite internet; weather forecasting accuracy diminishes, impacting disaster preparedness; and many financial and commercial systems relying on precise timing signals face major disruptions. The ability to mitigate failures and enable routine repairs and hardware upgrades of spacecraft in orbit hinges on advanced space robotics capabilities. However, the harsh environmental conditions of space—with challenging illumination, limited computational resources, and diverse motion regimes—make traditional computer vision and localization techniques developed for terrestrial applications inadequate for space applications. Space robotics demands perception capabilities that far exceed those of similar terrestrial systems. This research develops novel visual perception, localization, mapping, and planning algorithms to enable new capabilities for all future space missions such as failure mitigation, large flexible structure assembly, orbital debris removal, inspection, hardware upgrades, and many more. This research involves training of both graduate and undergraduate students. The results of this research are disseminated to the community by journal and conference publications, organization of invited workshops and seminar presentations, and by targeted exposure (press releases, interviews) to popular media. This research advances the state-of-the-art in computer vision and perception for space applications by addressing three critical limitations: first, existing feature detection methods struggle with the harsh illumination conditions of space; second current 3D reconstruction techniques fail to account for the dynamic orbital environment; and third, traditional simultaneous localization and mapping (SLAM) approaches cannot handle the diverse motion regimes of resident space objects (RSOs). To address these challenges, the research team develops a robust learning-based feature detection framework for space by using line features that exploit the inherent geometry of the target, along with multi-spectral imaging feature extraction. It also develops a novel dynamics-aware 3D Gaussian Splatting framework that incorporates relative orbital dynamics as physical constraints, enabling simultaneous state estimation and motion regime classification while maintaining physical consistency. The research team introduces a visibility-aware neural field representation that explicitly models observation uncertainty to drive information-theoretic view planning, therefore enabling autonomous space robots to systematically explore unknown objects through efficient observation sequences. Finally, the developed theory is experimentally tested and validated using Georgia Tech’s Autonomous Spacecraft Testing of Robotic Operations in Space (ASTROS) platform, a state-of-the-art spacecraft simulation platform, and also using a high-fidelity synthetic simulation environment. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.