CAREER: A Computer Vision for Global Biodiversity

About This Grant

Biodiversity is vital to the economy and future innovation opportunities, and we are currently witnessing an unprecedented loss of biodiversity. To better understand and hopefully mitigate this loss, networks of ground-level sensors, satellites, drones, and community scientists are deployed to collect natural-world data at unprecedented scales. There is valuable scientific information stored in these raw data, the vast majority of which are as-yet inaccessible due to the time and resources needed to process the data by small groups of relevant human experts. Computer vision (CV) will prove crucial to facilitate efficient extraction of scientific insights from quickly growing repositories of natural world imagery, but in order to realize the goal of global-scale, near-real-time biodiversity monitoring we must develop computer vision approaches keyed to challenges encountered in real world settings. This work formalizes and addresses cross-cutting limitations of current CV methods in the context of global-scale biodiversity monitoring, characterized in the following three research aims: (Aim 1) Robustly identify rare, visually similar, and even novel categories, all challenges separately for CV that co-occur in biodiversity data. We address this compounding challenge by augmenting limited training data and developing efficient active curation systems. (Aim 2) Adapt to new deployments and identify valuable data for specialized tasks. Biodiversity is non-uniformly distributed across the globe, and specialized models improve decision support. We introduce task-specific data selection as a specialization mechanism, and develop methods that adapt to new deployments over time while making optimal use of human effort. (Aim 3:) Share information and reason across modalities to fill data gaps and support scientific discovery. No single modality of biodiversity data captures the entire picture. We will develop data encoders that aggregate and share relevant information across modalities, and build on these encoders by developing interactive scientific AI agents that enable novel discoveries in data. These three research aims will be complemented by the development of cross-disciplinary educational programs that expand capacity at the intersection of CV and Ecology. Our proposed research innovations are necessary to enable robust, scalable CV deployments in ecological settings. Each aim outlined above will contribute towards our ability to deploy models with conservation partners and enable critical computer science/ecology collaborations. This combined effort will not only increase the utility of existing data and methods in the biodiversity domain, but will lead to advances in related application areas (including biology, astronomy, and neuroscience) as well as fundamental CV research challenges. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.