visual question answering (vqa)

Visual Question Answering (VQA) is a multimodal AI task where a system receives an image and a natural language question about that image and must produce an accurate natural language answer, requiring joint understanding of visual content and linguistic meaning. VQA demands diverse capabilities: object recognition (identifying what's present), attribute recognition (colors, sizes, materials), spatial reasoning (understanding relative positions and relationships), counting (how many objects of a type), action recognition (what entities are doing), commonsense reasoning (inferring unstated but obvious information), and reading (OCR for text visible in images). VQA architectures have evolved through: early fusion models (concatenating CNN image features with question embeddings and using MLP classifiers), attention-based models (using the question to attend to relevant image regions — stacked attention networks, bottom-up and top-down attention), transformer-based models (ViLT, LXMERT, VisualBERT — joint vision-language transformers with cross-modal attention), and modern large multimodal models (GPT-4V, Gemini, LLaVA, InstructBLIP — treating VQA as a special case of visual instruction following). Standard benchmarks include: VQA v2.0 (1.1M questions on 200K images with answers from 10 annotators), GQA (compositional questions requiring multi-step reasoning over scene graphs), OK-VQA (questions requiring external knowledge beyond image content), TextVQA (questions about text visible in images), and VizWiz (questions from visually impaired users photographing real-world scenes). VQA has been formulated as both classification (selecting from a fixed answer vocabulary — simpler but limited) and generation (producing free-form text answers — more flexible but harder to evaluate). Applications include visual assistance for visually impaired users, interactive image exploration, medical image analysis, educational tools, and robotic perception systems that need to answer questions about their environment.