Implementation partner-guided strategy to promote accuracy in ICU prognostication

About This Grant

Accurately predicting whether patients will live or die is an essential task in critical care medicine. Yet such predictions are often incorrect based on specific patient characteristics. We and others found that Acute Physiology and Chronic Health Evaluation (APACHE), Sequential Organ Failure Assessment (SOFA), and other prognostic tools overestimate the risk of death for certain groups of patients. Because the incidence of serious cardiac or respiratory critical illnesses is so high, the consequences of this error are significant for clinical decision making, research, and health policy. Given that APACHE II alone has been included in >10,000 studies, an entire medical literature may imperfectly measure the effectiveness of ICU treatments. Furthermore, the use of SOFA in ICU triage policies, such as during the COVID-19 pandemic, may propagate variability in health outcomes by incorrectly withholding life-saving ICU treatments from some patients. There are two key knowledge gaps that limit a solution including a lack of understanding of how measurement errors arise in these prognostic tools and the absence of ICU prognostic tools that are free of such errors. Filling these gaps will promote high-quality ICU care and access to it, as well as ICU research. Our goal is to develop a novel ICU mortality risk prediction model with accurate performance across all patient groups. To mitigate measurement error, we will combine state-of-the-art methods with the necessary supervision and contextual grounding provided by multidisciplinary advisors (critical care, bioethics, data science). Utilizing data from >100,000 patients broadly representative of the US population from 23 ICUs at Duke and Medical University of South Carolina, this project has three aims: (1) Identify mechanisms of systematic measurement error in commonly used ICU mortality risk prediction models; (2) Develop an accurate in-hospital mortality prediction model for all groups of critically ill patients; and (3) Conduct a real-world feasibility study of the new model within our electronic health record. The expected outcomes of this study, which addresses NHLBI’s key objectives to reduce variability in health outcomes and improve care by leveraging the power of data science, are two-fold. First, to our knowledge, this will be the first study to identify mechanisms of measurement error in current ICU mortality risk prediction models and to include variables that reflect the living situation of patients in ICU risk prediction—innovations that will serve as a conceptual foundation for future work. Second, the new model will have an immediate and substantial public health impact by promoting quality and representativeness in clinical care, research, and policy. Our proposal is innovative because it includes novel applications of model variability constraints and metrics; it also includes multidisciplinary advisors to ensure person-centered model development and evaluation. This study is feasible because of our team’s expertise and our rich research and clinical environment—all critical to navigating the complex sociotechnical context in which the proposal exists.