EAGER: Multi-Hop Reasoning with LLMs: A Structure-Guided Reasoning Approach

About This Grant

Large language models (LLMs) have been revolutionizing recent artificial intelligence research and applications. To empower complex reasoning and avoid hallucinations in special domain-focused problem-solving, LLMs need to incorporate updated and specialized data wisely and effectively. This project proposes a structure-guided reasoning approach for multi-hop, complex reasoning with LLMs. More concretely, it proposes to develop a retrieving-structuring-reasoning framework: (1) retrieving task-focused data and information, (2) structuring the retrieved data and knowledge, and (3) reasoning on the structured data and knowledge. Each task will need to explore the power of LLMs and develop efficient and effective LLM-integrated methods. The project will lead to the development of new algorithms for information retrieval, data and knowledge structuring, and structure-guided reasoning. The new methodologies generated will be broadly applicable across the fields of data science. Moreover, this research will support the cross-disciplinary development of a diverse cohort of doctoral and undergraduate students on both research and education at the University of Illinois. The technical aims of the project are divided into three thrusts. The first thrust, Retrieving, develops user- or task- query guided, theme-specific information retrieval to collect theme-specific documents, by developing effective methods for corpus-based, domain-specific multi-class text classification, knowledge-guided semantic indexing, and reinforcement learning-based information retrieval. The second thrust, Structuring, develops methods for automated extraction of entity and relation structures and construction of task-oriented knowledge structures, including causal structures, sequence structures, aspect structures, and multiple, interacting, function-specific knowledge graphs. The third thrust, reasoning, develops methods to evaluate and rank the extracted knowledge structures and feed such knowledge structures, together with retrieved theme-specific documents, into an LLM generation framework for effective reasoning, question answering, problem solving, explanation, and claim validation. The project will take the real-world, public scientific texts as typical datasets in experiments and it will result in the dissemination of shared data and benchmarks to the broader data science and AI communities. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.