Collaborative Research: Building AI Models to Help Middle School Students Interpret Science Diagrams

About This Grant

Representations such as diagrams, graphs, and charts are central to science and science education. However, learners often struggle with how to interpret science representations. The goal of this project is to develop, implement, and test a new AI assistant, the Representational Reasoning Assistant (RRA), to help middle school students interpret representations in their science classrooms. The AI assistant will draw on cutting edge Generative AI technologies to engage learners in conversations about the representations assigned by their teachers, ask the learners guiding questions, and offer suggestions about where to look in order to make sense of the representations. A key component of the design is to enable teachers to modify the AI assistant easily based on knowledge of their students and on the tasks which they set as priorities for their students. The project will help advance interdisciplinary research and practices in AI, computer science, learning sciences, and STEM learning. Throughout the three years of the project, teachers and students will be recruited from urban, suburban, and rural schools. The sequence of research and development activities reflects an integrated effort between the learning sciences and computer science teams. The project consists of iterative cycles of exploration, development, pilot and model refinements of the AI assistant, focusing on the types of representations teachers use in science activities and the types of feedback they give to students. Multimodal Large Language Models (MLLMs) will be adapted to be visually focused, supportive of pedagogical intent for young learners, and include innovations in rapid training to support a wide range of classroom topics and contexts. Early rounds of the piloting will gather teacher feedback on initial models and versions of the AI assistant. The AI assistant interface will then be fine-tuned based on teachers and students' feedback as well as measurements of students' engagement and learning. This project is funded by the Research on Innovative Technologies for Enhanced Learning (RITEL) program that supports early-stage exploratory research in emerging technologies for teaching and learning. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.