Research Infrastructure: NSF Mid-scale RI-2: Open Multimodal AI Infrastructure to Accelerate Science

About This Grant

Language models with billions of possible adjustments and trained on trillions of words are now powering the fastest-growing computing applications in history. Large Language models (LLM) are built using massive amounts of text, usually obtained by pulling data from multiple sources on the internet. Recent advances enable these models to process other kinds of data, including images, graphs and tables. Models with these abilities are known as multimodal LLMs. The best-performing LLMs currently deployed are proprietary, so their parameters, training data, code and documentation are not openly available. Thus, most artificial intelligence (AI) scientists cannot study, experiment directly with, or improve these state-of-the-art models. This project – Open, Multimodal Artificial Intelligence (OMAI) - will provide infrastructure in the form of a suite of powerful, well-documented, up-to-date, open models, and open-source interfaces designed for scientific work. Scientists will be able to access the models, use discipline-specific data and optimize the models. The project empowers researchers, provides documentation to accelerate research and education, and has an active program in early-career training to advance US economic and scientific competitiveness. In addition, opportunities provided through partnerships with a range of institutions and programs will enhance training. The long-term plan is to make the infrastructure available as a low- or zero-cost service to the research community in a manner like open-source code repositories and science-focused digital libraries, to maximize usage. The OMAI research infrastructure consists of a series of open, multimodal language models kept up to date with recent scientific publications and open-source application programming interfaces that enable scientists to use, expand, and modify those models. It addresses priorities set forth in the White House AI Action Plan (https://www.whitehouse.gov/wp-content/uploads/2025/07/Americas-AI-Action-Plan.pdf) to accelerate AI-enabled science and ensure the United States is producing the leading open models. The infrastructure aims to accelerate scientific discovery across disciplines ranging from materials to protein function prediction and weather models. It will also enable new understanding and improvement of future LLMs while contributing to the development of a well-trained workforce capable of building, customizing, and maintaining such models. In allowing researchers to fine tune the models, researchers can optimize performance and understand design decisions that influence training speed and stability, which can impact the short-term and long-term economic costs of LLM development. The project emphasizes reproducibility, transparency, open data, open and evolving evaluations, multimodality, and scientific use-cases. It will enable a broad population of scientist-users across all disciplines to use and adapt artificial intelligence models to their own needs and lays the foundation for future research in AI for science. By supporting work in these novel, critical research areas, OMAI can ultimately benefit both science and the public. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.