Developing a nonvisual, tangible interface for the Scratch educational programming environment

About This Grant

This Developing and Testing Innovations project will continue the development of a nonvisual interface for the Scratch educational programming environment. Drag-and-drop programming environments like Scratch have done much to lower the barrier to learning and increase engagement for a variety of students. Such resources support K-12 students' exploration and preparation for careers in computer science, and in many other science, technology, engineering and mathematics fields that require programming skills. This project will continue the development of a tangible, nonvisual programming interface to allow blind and visually impaired (BVI) students to fully and independently participate in the same Scratch learning opportunities as their sighted peers. A key component of the approach is to translate visual aspects of Scratch into the haptic domain, focusing specifically on elements known to increase engagement and lower hurdles to learning programming and taking into account the differences in how the haptic and visual sensory systems process information. In addition, resources will be developed for teachers who may have students using the nonvisual interface alongside those using the visual interface for Scratch, including provision of some best practices. The goals of the project are threefold. First, the project will continue to develop the nonvisual interface for Scratch through a) using user feedback and observations of a two-day programming camp for BVI middle school students held at the end of the previous project term to improve the system design, b) automate components of the interface previously implemented using Wizard of Oz methods, and c) add components designed to increase engagement of a wide variety of learners, in a similar manner to visual Scratch. Second, the project will develop teaching resources, including curricula, support materials and training videos for K-12 teachers; these resources will incorporate how to integrate instruction for nonvisual and visual interfaces, and best practices for paired programming with the nonvisual interface. Third, the project will develop mentoring and educational guidance resources for students using the nonvisual interface. All of these components will be developed through the use of universal and participatory design techniques, as well as evidence-based design. Multiple five-day camps for mixed groups of BVI and sighted students will be held at collaborating science centers to provide an objective measurement of the effectiveness of the interface and teaching materials developed, and provide insight into the requirements for widespread deployment. This project is funded by the Innovative Technology Experiences for Students and Teachers (ITEST) program, which supports projects that build understandings of practices, program elements, contexts and processes contributing to increasing students' knowledge and interest in science, technology, engineering, and mathematics (STEM) and information and communication technology (ICT) careers. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.