Knowledge-based VQA systems retrieve documents and answer from them. My work asks two questions: do our benchmarks measure what we think they measure, and what should a system do when the retrieved evidence narrows the answer without settling it? The current project builds an agentic multi-hop retriever over knowledge graphs.
Accuracy on E-VQA and InfoSeek only measures knowledge-grounded reasoning if answers are derivable, questions are well-posed and images need real disambiguation. We audit both benchmarks, repair them, and add distractor-entity variants; model rankings change under both.
Retrieved evidence often narrows the answer without fixing it. We formalize this resolvable insufficiency, where partial evidence plus one more step is enough, and separate it from cases where a system should abstain.
Biomedical text states what a drug treats far more often than when it must not be used. We quantify this contraindication gap in the source text and in the language models trained on it.
Many questions need a chain: identify the entity in the image, cross to a related entity, then read the answer there. Documents often name only part of that chain. I am building an agent that decides at each hop whether to expand the search, check the candidate against the image, or stop and abstain.