Comparing Decision Makers: Theory and Implications for AI

About This Grant

This project develops a new method for comparing human and artificial intelligence (AI) decision-making, generating insight on how to combine human and AI input in high-stakes decisions like medical diagnoses. While researchers can easily observe an AI tool’s prediction process, human decision-making is more complicated. For example, a radiologist’s diagnostic decisions may reflect both their judgments about the presence of cancer and their assessments of the cost of an incorrect diagnosis, but researchers often only observe their final decision. This project develops a method for comparing human and AI decision-making that accounts for complex human preferences and judgements caused by factors that the researcher cannot directly observe. The researchers apply these methods to a radiology setting, using data from a large clinical trial to compare an AI tool’s predictions of lung cancer to radiologists’ diagnoses. This research helps determine the optimal use of AI in high-stakes decisions by answering questions like the following: What valuable information do humans infer that AI assessments miss? What are the intricacies of human preferences during decision-making, and how do these intricacies affect the quality of human decisions compared to AI decisions? This research generates insights that can be used to design better AI-supported decision systems in radiology and other high-stakes settings, leading to public health benefits, a deeper understanding of human and AI decisions, and more responsible use of AI. The project makes three methodological and empirical contributions. First, the researchers show that commonly used approaches for comparing human experts against predictive algorithms are biased in favor of algorithms if the preferences of human experts vary due to factors not observed in the data. Second, the researchers construct bounds on the quality of human expert predictions in the presence of unobserved preference heterogeneity and identify in which specific cases the human expert observes useful private information. Third, the researchers characterize the distribution of preferences across human decision-makers and cases. In an application, the project uses data from the National Lung Screening Trial to compare an AI tool for lung cancer detection against radiologists, empirically quantifying the importance of private information observed by humans that is missed by the AI tool and the extent of preference heterogeneity across cases and radiologists. Using these results, the researchers consider two practical problems: the delegation of cases to humans or AI based on case characteristics, and the design of an automated decision-making tool that aligns AI predictions with the newly determined bounds on human decision-maker preferences. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.