CAREER: Integrating Conformal Prediction into Machine Learning for Provably Safe Deployment

About This Grant

Advances in machine learning (ML) have opened up new possibilities for accurate predictions in various fields. To ensure the reliability of these predictions, especially in high-stakes decision-making scenarios, theory and tools need to be developed that provide confidence intervals. This research project aims to create innovative methods for quantifying uncertainty in complex systems, allowing for more informed and confident decision-making. One key aspect of this work is designing efficient collaboration between humans and machines, where artificial intelligence (AI) produces a set of possible solutions and human experts select the best option. For instance, in medical diagnosis, these tools will enable doctors to identify potential health risks by generating a short list of likely diagnoses, with the correct answer guaranteed to be included among them. Through a five-year program combining cutting-edge research, education, and community engagement, the researcher will develop novel algorithmic and theoretical frameworks to improve predictive accuracy and trustworthiness across various applications. To share the results broadly, the researcher will publish research papers, present at conferences, and provide tutorials on uncertainty quantification and safe ML deployment. This project aims to create a new framework that seamlessly integrates conformal prediction principles into ML model training procedures. This integrated approach will enable the production of small uncertainty sets with rigorous guarantees, facilitating the safe deployment of ML models and efficient human-ML collaboration. To achieve this goal, the project will focus on three interconnected research areas. First, develop calibration methods that can provide reliable assessments of confidence intervals using two complementary metrics: conformity scores from trained classifiers and label ranks. Second, establish principled training objectives and optimization algorithms for conformally training deep neural network classifiers. Third, develop conformal calibration techniques tailored to large language models (LLMs) and optimize their fine-tuning procedures using a novel framework. These methods will address the unique challenges posed by LLMs while ensuring robust performance in high-stakes applications. This research will be applied to real-world domains, including adaptive experimentation for design optimization, cybersecurity, health monitoring, smart agriculture, and elderly care. The team will collaborate closely with domain experts to ensure that our solutions meet the needs to tackle pressing societal problems. The developed algorithms will be freely available through open-source software, enabling widespread adoption by academia and industry. The team also plans to engage with the broader community through education and outreach initiatives, including a novel Ambassador program to encourage underrepresented minority students in computer science, a short summer course on uncertainty quantification for engineers and scientists, and a partnerships with existing programs to recruit students in computer science. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.