Visual question answering for biology

Vision-language models are increasingly asked to read biological images, but a single VQA score hides which abilities they have. Bio-VQA splits biological visual understanding into four components with their own data, graded metrics and baselines, and evaluates 18 open-weight VLMs against an image-blind floor and a specialist ceiling.

Project page

Bio-VQA's four components: hierarchical taxonomic reasoning, fine-grained attribute grounding, quantitative spatial reasoning and individual re-identification.

WACV 2027First author

How Finely Can Vision-Language Models See Biology? Bio-VQA: A Benchmark for Taxonomic, Morphological, Spatial and Individual Reasoning

S M Rayeed, Qian Ma, Charles V. Stewart, Yao Ma
IEEE/CVF Winter Conference on Applications of Computer Vision (WACV 2027)

The largest biological VQA benchmark: 1.86M questions over 217K images from 11 public datasets, spanning hierarchical taxonomic reasoning, fine-grained attribute grounding, quantitative spatial reasoning and individual re-identification. Across 18 open-weight VLMs, general visual skills do not transfer: the best model counts animals exactly on only 41.7% of images, and 15 of 18 models measure traits no better than an image-blind baseline.

Project page Paper Code

Hierarchical taxonomic reasoning: one image is asked its identity at eight ranks, from phylum down to common name, plus a four-way multiple-choice common name.

Quantitative spatial reasoning: eight tasks including counting, OCR, relative position, viewpoint, absolute distance and localization.

Other research areas

Knowledge-based VQA and retrieval Fine-grained visual recognition Vision datasets and detection pipelines Instance-level re-identification Sign language recognition