AI-Based Screenomic Analysis of Digital Experiences and bidirectional Links to Psychopathological Symptoms

About This Grant

Project Summary Adolescent social media use has risen dramatically. This rise has occurred alongside with increasing rates of depression, anxiety, and other psychopathological symptoms. However, current research lacks the understand- ing of which specific online experiences drive mental health risks. This project pioneers the use of AI-based screenomic analysis to transform over 38 million smartphone screenshots from 154 adolescents into structured records of digital experiences, enabling the first scalable, causal study of risky online exposures and their bidirec- tional links to psychopathological symptoms. Towards this objective, we propose the following specific aims: (1) we will develop computational pipelines to detect event-level risky exposures—individual encounters with harm- ful content (e.g., violence, suicide, body image) or risky social interactions (e.g., cyberbullying, peer pressure). Vision-language models (VLMs), fine-tuned and personalized using few-shot adaptation, will integrate visual and textual cues to classify these exposures. These event-level data will be linked to biweekly measures of anxiety and depressive symptoms using Random-Intercept Cross-Lagged Panel Models to assess reciprocal causality. (2) We will move beyond single events to model sequential risky digital exposures—chains of online behaviors that may amplify risk (e.g., influencer viewing → body image discussion → late-night browsing). Multi-image VLMs and sequential pattern mining will identify common trajectories, and Marginal Structural Models will test whether these digital patterns predict symptom escalation, or vice versa. By advancing from broad “screen time” metrics to fine-grained digital biomarkers, this research will establish a mechanistic understanding of how online experiences shape mental health in adolescence. The resulting computational tools and insights will lay the foun- dation for precision monitoring and prevention strategies, informing policy, clinical interventions, and youth digital well-being initiatives.