Navigation assistance using multimodal LLM and computer vision
NEI - National Eye Institute
About This Grant
Project summary Navigation can be a significant challenge for blind and visually impaired (BVI) individuals, often requiring assistance, for instance, remote sighted assistance via Aira app. Existing navigation aids, including mobile apps, have limitations in accuracy and reliability, providing only partial solutions. In general, GPS-based apps are not suitable for micro-navigation (short-range navigation like finding building entrances). This is known as the “last 30 feet” problem. We propose a comprehensive, accessible end-to-end navigation solution for BVI travelers, encompassing both macro-navigation and micro-navigation segments. This solution will be implemented as mobile apps that can run on smartphones. In this project, we will first develop and validate three key modules required for the solution. These modules, which can be packaged as standalone assistance apps for BVI navigation, include (1) All Walk: Plans end-to-end navigation directions using map services and guides travelers with sub-foot precision on sidewalks; (2) NavGPT: Provides micro-navigation directions using specialized multi-modal language model; (3) CamGuide: Tracks targets suggested by large vision language model and guides BVI individuals towards them with auditory feedback. AI models will be custom-trained, including a multi-modal vision language model for scene understanding and computer vision models for object recognition. The training dataset will be collected from actual navigation scenarios of BVI individuals. In the second phase, the three modules will be integrated into a comprehensive navigation assistance app, SmartWay. Throughout the entire project, we will collaborate with the research team from the Carroll Center for the Blind to ensure the apps meet accessibility guidelines for both blind and partially sighted users. We will also conduct field evaluation studies with BVI participants, comparing our apps with existing navigation solutions such as Google Maps and Aira.
Grant Summary
Navigation assistance using multimodal LLM and computer vision is a NEI - National Eye Institute grant providing up to $879K for university, nonprofit, healthcare org. Applications are due 2031-04-30 (open). Check eligibility and apply with FindGrants.
Focus Areas
Eligibility
How to Apply
Up to $879K
2031-04-30
- 1Confirm your organization is eligible for Navigation assistance using multimodal LLM and computer vision from NEI - National Eye Institute, checking organization type, location, and any population or project requirements.
- 2Gather the required documents and information, including your organization details, project plan, and budget figures.
- 3Draft your application narrative and budget addressing the funder's priorities and review criteria. FindGrants can draft each section for you to review and edit.
- 4Review every section against the requirements checklist, then export a submission-ready application pack and submit it to NEI - National Eye Institute before the deadline.
Don't want to draft it yourself?
We'll draft the complete application against NEI - National Eye Institute's requirements, run a quality review, and email you a submission-ready PDF plus an editable Word doc within 5 business days. Most orders deliver in 24-48 hours. Flat $399, any grant size.
AI Requirement Analysis
Detailed requirements not yet analyzed
Have the NOFO? Paste it below for AI-powered requirement analysis.
Navigation assistance using multimodal LLM and computer vision: Frequently Asked Questions
Who is eligible for the Navigation assistance using multimodal LLM and computer vision?
Navigation assistance using multimodal LLM and computer vision is offered by NEI - National Eye Institute and is generally open to university, nonprofit, healthcare org. It is open to organizations nationwide unless the funder specifies otherwise. Review the specific eligibility terms before applying, since funders set their own requirements around organization type, location, and the population or project being served.
How much funding does the Navigation assistance using multimodal LLM and computer vision provide?
Navigation assistance using multimodal LLM and computer vision provides up to $879K per award from NEI - National Eye Institute. Actual award sizes depend on the scope of your project, available program funds, and the number of applicants, so build a budget that reflects realistic, allowable costs rather than the maximum figure.
When is the Navigation assistance using multimodal LLM and computer vision deadline?
Applications for Navigation assistance using multimodal LLM and computer vision are due 2031-04-30 (open). Because deadlines can change, verify the date with the funder, NEI - National Eye Institute, and give yourself enough time to prepare a complete, competitive application before the close date.
How do you apply for the Navigation assistance using multimodal LLM and computer vision?
To apply for Navigation assistance using multimodal LLM and computer vision, confirm your eligibility, gather the required documents, and prepare a narrative and budget that address the funder's priorities. FindGrants guides you step by step and can draft each section, then exports a submission-ready application pack for this grant from NEI - National Eye Institute.