CPS: Small: Confidently Safe Autonomy Under Anomalies

About This Grant

The behavior of cyber-physical systems (CPS) powered by Artificial Intelligence (AI) is becoming increasingly impactful, complex, and potentially dangerous. Therefore, it is critically important to monitor the safety of the deployed AI-enabled CPS. However, this monitoring can be disrupted and rendered ineffective due to anomalous data that arises due to open environments and sophisticated AI behaviors. How much do these anomalies degrade the safety of the CPS? To answer this question, this project will devise an approach to computing the predictive confidence in the safety of a CPS experiencing one or several anomalies. This approach will be validated on small-scale autonomous racing cars and autonomous underwater vehicles. If successful, this research will result in superior safety monitoring capabilities and, thus, support continued adoption and deployment of advanced AI in automated CPS of great societal importance. Despite recent advances in anomaly detection, there is little connection between anomaly severity and safety violations in a CPS, beyond vague statistical correlation. This project seeks to close this gap by designing a general methodology to compute safety confidence in AI-enabled CPS undergoing anomalous behavior. The key insight is that the state-of-the-art anomaly measures can be aligned with the typical CPS components to inform online safety prediction. Leveraging this insight, this project will develop a collection of modular, meaningful, and safety-relevant anomaly scores for perception, dynamics, and control components of a typical closed-loop CPS. These scores will then be used to inject uncertainty into safety monitoring using symbolic functions that provide formal guarantees of calibrated prediction of safety confidence. As a result, this methodology promises to make autonomous CPS aware of how anomalies impact their safety. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.