Exploration and adaptation of experience sampling to understand the impact of artificial intelligence tools on the work of teaching

About This Grant

Despite popular discourse about how generative artificial intelligence (AI) tools might increase efficiencies in education, it is yet to be determined how--or even if--AI tools for teachers are having that envisioned effect. There is currently a baseline assumption that AI should increase teacher efficiency through offloading. However, there is limited research to support such claims, and there is reason to believe that AI could increase workload or cause unexpected shifts in teacher responsibilities due to the unique demands of classroom teaching. Researchers need new methodological approaches to understand generative AI's impact on teacher experience and daily work. By exploring and refining research instrumentation to study how AI affects teaching, this project will inform efforts to improve educational practice and ensure that new technologies support rather than burden teachers. The resulting instrumentation will be made publicly available to support broader investigations into how emerging technologies shape the teaching profession. The goal of this ECR: Level I project is to examine, adapt, and deploy instrumentation that can help researchers investigate generative AI's impact on teacher work and generate necessary information to guide future decisions. Most immediately, the instrumentation will enable closer scrutiny of assumptions about teacher efficiency with AI. The project centers on experience sampling method (ESM), a technique originating from psychology that involves repeated brief random digital surveys to capture what people are doing, when they are doing it, in what context, and how they are thinking about it. Although advocated for in education research, ESM remains largely underexplored for studying teachers. With widespread access to mobile devices, ESM is more feasible than in previous decades, but requires intensive design and testing to ensure it fits the realities of classroom teaching. Over three years, this pilot study will adapt ESM for use with teachers and, in parallel, examine the extent and nature of efficiency changes as a result of a partnering school district's commitment to supporting teacher use of generative AI. The district serves over 10,000 K-12 students, offering a meaningful test case for understanding AI-related changes in practice. The research will focus on two key questions: (1) What attributes and parameters do teachers consider in their perception of their work allocation, and how could those be sampled given unique challenges associated with the work and schedules of classroom teaching? and (2) How effectively does digital ESM obtain a record of teachers' activities as sampled in comparison to human observation and retrospective recall? The work will begin with collaborative design with teachers to determine appropriate approaches and queries for ESM that respect privacy and workload. It will then compare ESM data to other observational techniques to evaluate the data quality of this new approach. The resulting instrumentation and validated protocols will contribute new tools for education researchers to test assumptions about technology's impact on teacher work, advancing the field's interest in expanding the use experience sampling methods for studying educational practice. This project is funded by the EDU Core Research (ECR) and Innovative Technology Experiences for Students and Teachers (ITEST) programs. It supports the ECR program, which emphasizes fundamental STEM education research that generates foundational knowledge in the field. It supports ITEST program goals to build understandings of practices, program elements, contexts and processes contributing to increasing students' knowledge and interest in STEM and information and communication technology (ICT) careers. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.