Collaborative Research: SCH: Personalized Watch-based Fall Risk Analysis and Detection with Cross Modal Learning

About This Grant

Falls are a significant cause of morbidity and mortality in the elderly. A robust and low-cost solution for the estimation of fall risk and detection of falls will allow seniors to live independently and reduce medical costs due to falls. Wearable devices have been developed to detect “hard falls”, namely falls that cause injury. However, many falls in the elderly do not cause physical injury (“soft falls”). These occur in association with weight transfer activities such as turning and sit-stand transitions. Indeed, the ability to control the position and movements of the trunk (“core”) is essential for coordinating the movements of the limbs during weight transfer. The goal of this project is to combine real-world limb-core dynamics of an individual with data collected by accelerometer via a commodity wristwatch and a cell phone on the opposite hip to improve the detection of hard and soft falls. A personalized fall risk analysis and detection model will be created for each user via real-time learning of the limb-core dynamics using state of the art machine learning algorithm. We will also assess the perceptions and preferences of elderly patients using this technology and evaluate their attitudes towards continuous data collection and sharing of health data for improved health. The software system, the real-world gait and weight transfer movement and the associated accelerometer data will be made freely available to any institution, investigator or research student interested in the study of machine learning on health conditions as well as on fall risk and analysis. This project will train graduate and undergraduate students in technical skills (machine learning, wearable technologies and data analysis skills) as well as in people skills for working with the elderly who live in long-term care facilities. While numerous fall detection devices incorporating artificial intelligence (AI) and machine learning algorithms have been developed, this project focuses on personalizing fall risk detection. This project will explore the use of kinematic measurements of an elderly individual’s movements associated with weight transfer to enable multi-task and multi-modal machine learning algorithms to personalize fall risk detection. A small-sample-based deep learning algorithm optimized to incorporate individual kinematic characteristics using multi-task and multi-modal learning frameworks is developed. Second, the team will analyze the movement transitions captured by the Azure Kinect system in order to identify relationships between the accelerometer data and the complete skeletal frame with an emphasis on the limb-core dynamics. Specifically, our goal is to determine whether or not Generative Adversarial Networks (GANs) can be used to augment missing modality from a small amount of body motion data, smartwatch and phone acceleration data collected directly from elderly participants who are at most risk of falling, namely those living in an assisted living center. Finally, we will evaluate the perception and attitudes of the elderly participants towards the continuous use of wearable devices for fall risk analysis and detection. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.