CAREER: A Principled Framework for Multi-Task Representation Learning for Scalable, Decentralized, and Safe Sequential Decision-Making

About This Grant

Real-world systems are composed of interconnected entities that collaborate to perform diverse yet interrelated tasks, often requiring sequential decision-making. Over the past decade, multi-task learning has emerged as a powerful paradigm for collaborative learning, significantly enhancing efficiency while enabling privacy-preserving knowledge sharing. One of the most promising approaches to learning-based control is dynamic sequential learning, such as reinforcement learning, which learns through interactions with the environment. However, reinforcement learning faces three critical challenges in real-world dynamical systems: data scarcity and heterogeneity, scalability and communication efficiency, and safety. Moreover, achieving provable guarantees in joint learning often requires leveraging underlying problem structures. This CAREER project will develop a unified approach to multi-task representation learning by leveraging the shared representations to offer a viable solution to these challenges, enabling privacy-preserved joint learning in dynamic environments. Research and education will be synergistically integrated to train students in the interdisciplinary field of data science and control, addressing the pressing need for skilled workforce development in this emerging area of societal importance. The central objective of this project is to develop provable methods for multi-task representation learning in bandit and reinforcement learning settings. At its core is a novel algorithm, (de)centralized Alternating Gradient Descent and Minimization (AltGDmin), designed to address the challenges of non-convex, under-sampled, and constrained problems. This approach enables fast and federated representation learning while effectively tackling data heterogeneity, scalability, communication efficiency, and safety concerns. The project technical plan includes three research thrusts together with several validation activities and an integrated education plan. The first goal is to create a provable few-shot personalized federated multi-task representation learning framework for bandit and reinforcement learning. The second goal is to develop a fully decentralized, federated multi-task representation learning framework for bandit and reinforcement learning over a networked architecture among agents. Our approach will eliminate the central server, enhancing scalability, communication efficiency, and robustness by removing single points of failure. The third goal is to develop an innovative safe multi-task representation learning framework to learn optimal policies while incorporating the safety constraints. The main application domains of interest are control and automation in smart farms, which will guide the problem formulation and validate the algorithms using real-world implementation, testing, and numerical experiments. The overarching goal of the integrated education plan is to provide a pathway for K-12 to college students to receive rigorous math training and hands-on experience, including coding skills for ML-based intelligent system design. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.