CAREER: Advancing Efficient Global Optimization of Extremely Expensive Functions under Uncertainty using Structure-Exploiting Bayesian Methods

About This Grant

Mathematical optimization is the process of maximizing a performance or quality indicator by identifying the best possible value among the set of all feasible options. Optimization problems arise in virtually all human endeavors related to decision making including engineering, economics, sustainability, healthcare, and manufacturing. Instances of such optimization problems are particularly challenging to solve whenever evaluating performance and/or testing for feasibility requires an expensive simulation or experiment whose results may be corrupted by random errors. Bayesian optimization (BO) is a class of machine learning-based optimization algorithms that has recently been shown to achieve state-of-the-art performance in several important applications from this problem class such as in deep machine learning, validation of expensive simulators, and material and drug design. However, traditional BO methods treat the mathematical functions that model performance and feasibility as black boxes with unknown structure, which sets a fundamental limit on their computational efficiency. This observation is the key motivation for this research project, which looks to overcome these efficiency barriers via the development of new algorithms that exploit known problem structures within the Bayesian framework. These novel capabilities will be applied to three unsolved problems currently impacting society: (1) identifying unknown mechanisms in cellular decision-making processes for biomanufacturing; (2) discovery of new sustainable and economical lithium-ion battery electrode materials; and (3) real-time energy minimization in industrial heating, ventilation, and air conditioning (HVAC) systems. In addition, the project looks to tightly integrate research and educational activities through the development of interactive workshops and games related to decision science, which will be made accessible to the public. Through collaboration with local educators, planned outreach activities also will provide K-12 students from underrepresented communities with opportunities to learn about decision science. The proposed optimization methodology is inspired by the principle of grey-box modeling, which states that one should avoid learning what is already known when applying machine learning methods. The investigator conjectures a significant reduction in experimental and/or computational effort can be obtained in practice over traditional Bayesian optimization (BO) methods by properly leveraging prior (or domain) knowledge, which is almost always available in practice. Since prior knowledge can come in many diverse forms, the proposed research will focus on some of the most common and important examples. The three specific research aims are: (1) optimizing with hybrid physics-based and data-driven models given noisy and incomplete datasets; (2) optimizing with constrained multi-fidelity models that fuse data from a collection of heterogeneous sources of variable accuracy and cost; and (3) scaling to high-dimensional and sparse data problems through the incorporation of non-myopic and graph-structured formulations. The proposed research aims to promote convergence of statistics, machine learning, optimization, and process systems engineering. More broadly, the improved methods developed as a part of this research project will allow practitioners to solve a wide range of grey-box optimization problems with greater speed and accuracy. Planned outreach activities include educating K-12 students about decision-making under uncertainty via interactive workshops and games, incorporating new data-driven optimization material into the chemical engineering curriculum, and organizing cross-disciplinary professional workshops on the potential significance and impacts of cutting-edge BO technology. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.