Advancing nonparametric regression: from convex multi-index and Bayesian models to BART and neural networks

About This Grant

This project addresses fundamental challenges in statistical modeling, to develop more accurate and reliable methods for analyzing complex data. As data becomes increasingly central to scientific discovery, economic prosperity, and national security, the need for advanced statistical tools is paramount. This research will create new techniques in nonparametric regression, a field of statistics focused on fitting models to data without pre-supposing the relationship's form. It confronts three recurrent obstacles in analyzing large datasets -- curse of dimensionality, ad-hoc tuning choices, and the tension between flexibility and interpretability -- by developing principled regression and density-estimation tools, thereby improving our ability to interpret complex information. The work forges new links between shape-constrained nonparametric methods and neural networks, adapts ideas from image processing to statistics, and also unites frequentist and Bayesian thinking through simple, intuitive priors. The development of these methods will have wide-ranging benefits in many applied fields. Furthermore, this project will contribute to the education and training of the next generation of statisticians and data scientists, ensuring that the nation remains at the forefront of this critical field. The investigator will develop a suite of novel approaches to nonparametric regression. One area of focus is a new shape-constrained method for multi-index convex regression, which is designed to alleviate the curse of dimensionality and has close connections to single hidden-layer neural networks. Another key component of the research involves the systematic study of Total Generalized Variation (TGV) regularization for regression and density estimation problems that have both smooth and non-smooth components, a common challenge in fields like image processing. The project will also investigate the properties of the log-concave maximum likelihood estimator, with the aim of proving its suboptimality in high dimensions under the total variation distance. Additionally, the research will explore Bayesian approaches with innovative priors, such as those based on Cauchy processes, to model complex regression relationships and address the issue of tuning parameter selection. Finally, the research will develop Bayesian methods for mixed-derivative constrained regression, leading to the creation of an additive regression tree and piecewise linear fits for greater flexibility in multivariate settings. These research thrusts will be pursued through a combination of theoretical analysis and computational experiments, to produce practical and principled solutions to outstanding problems in nonparametric regression. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.