S M Rayeed

S M
Rayeed

I build benchmarks and methods that test what vision-language models actually see, identify and know about the living world.

PhD student in Computer Science, Rensselaer Polytechnic Institute
Computer Vision Lab, funded by the NSF Imageomics Institute

CV Email Scholar GitHub LinkedIn

Open to research internships for summer 2027.

elytra lengthelytra widthCarabus nemoraliselytra lengthelytra widthChlaenius aestivuselytra lengthelytra widthCicindela sexguttataelytra lengthelytra widthPterostichus melanariuselytra lengthelytra widthDicaelus purpuratuselytra lengthelytra widthAmara aeneaelytra lengthelytra widthScaphinotus viduuselytra lengthelytra widthPoecilus lucublanduselytra lengthelytra widthHarpalus pensylvanicuselytra lengthelytra widthMecyclothorax konanuselytra lengthelytra widthBlackburnia kilaueaelytra lengthelytra widthSynuchus impunctatus
Select a specimen to measure its elytra.Family Carabidae
Portrait of S M Rayeed

About

I am a fourth-year PhD student in Computer Science at Rensselaer Polytechnic Institute, advised by Charles V. Stewart and Yao Ma, and funded by the NSF Imageomics Institute.

My work sits where computer vision meets biology and knowledge. I benchmark vision and vision-language models on fine-grained species identification, built BioVQA to separate what models know from what they see, and audit and improve knowledge-based VQA, where retrieval systems must decide whether the evidence they found is enough to answer.

I also curate the data this research runs on: a continental-scale ground beetle dataset in Nature Scientific Data, the pipelines that turn collection trays into AI-ready records, and the Grevy's Zebra Census Dataset. Before RPI, I built BdSL47, the first complete depth-based Bangla Sign Language dataset, at the Islamic University of Technology.

News

  1. BioVQA is accepted to WACV 2027, Datasets and Benchmarks track.
  2. Reviewing for the KDD 2027 AI for Science track, Communications Earth & Environment and PLOS ONE.
  3. Our continental-scale ground beetle dataset is accepted at Nature Scientific Data.
  4. Our audit of knowledge-based VQA benchmarks appears at ECCV 2026.
  5. Two papers at the NeurIPS 2025 Imageomics workshop: BeetleFlow and Mind the (Data) Gap.
  6. Presented the fine-grained beetle taxonomy benchmark at the ICCV 2025 CV4E workshop.
  7. Two papers at the CVPR 2025 CV4Animals workshop: BeetleVerse and multiview track clustering for re-ID.

Research

All research areas
Curated collectionsBeetlePaloozaBeetlePUUMNHM-CarabidsTransfer holds between lab collectionsIn the fieldI1MC: varied pose, occlusion, cluttertrain37%performance drop fromlab-trained modelsViLT + MLP: 97% genus, 94% species accuracy on merged data.50% of the training set gives nearly the same accuracy.

Fine-grained taxonomic identification

Which vision models can tell 1,769 beetle species apart, how little data they need, and where they break.

3 papersExplore this area
AuditRepairAugmentQ: When was it built?A: 1889Knowledge base: no dateQuestion: which building?Answers unsupported by theknowledge base:22%of InfoSeekUnderspecified questions:59%E-VQA47%InfoSeekQ: When was the iron latticetower in this photo built?A: 1887 to 1889Evidence: KB passage 3Derivable answers, clearquestions. Human accuracy:92.9E-VQA91.5InfoSeekdistractor entityA second entity in the image.InfoSeek retrieval R@1:43.514.7QA accuracy of strong systemsroughly halves (IBA 40.1 to 21.4)Model rankings change once benchmarks are repaired and augmented.

Knowledge-based VQA and retrieval

Benchmarks that measure real knowledge-grounded reasoning, and retrieval systems that know when evidence is not enough.

3 papers, work in progressExplore this area
Tray photographIterative detectionGrounding DINO + SAM; found boxes aremasked and the tray is re-detected atlower thresholds, 0.30 down to 0.10VLM check:any left?yes: flag for recheckCropped and ordered123456Scale bar and color cardlocated with MoondreamNEON recordsMetadata validatedagainst the databaseCuration errors returned to the biorepository65,236specimens3,649trays518species96%tray-level detection

AI-ready biodiversity data

Turning museum drawers and NEON traps into validated, machine-learning-ready datasets.

4 papers, 2 datasetsExplore this area
21 keypoints, x, y and depth63featuresANN, 4 hidden layersঅআইউএওকখগঘচছজঝটঠ০১২৩৪৫৬৭47 classes47,000images10signers37 + 10letters and digits97.84%F1 score

Sign language recognition

Depth-based recognition of Bangla Sign Language, and the first complete open dataset for it.

2 papers, 1 datasetExplore this area

Selected publications

All 14 publications
ConferenceJournalWorkshopFirst author
  1. WACV 2027First author

    BioVQA: A Benchmark for Biological Question Answering across Taxonomic, Fine-grained and Spatial Reasoning

    S M Rayeed et al.
    IEEE/CVF Winter Conference on Applications of Computer Vision (WACV 2027), Datasets and Benchmarks track
    Project pagePaperCode
  2. ECCV 2026

    Identifying and Resolving Pitfalls of Knowledge-Based VQA Benchmarks: Auditing, Repairing, and Augmenting

    Qian Ma, S M Rayeed, Charles V. Stewart, Qiong Wu, Yao Ma
    European Conference on Computer Vision (ECCV 2026)
    PaperarXivCode
  3. Scientific Data 2026First author

    A continental-scale dataset of ground beetles with high-resolution images and validated morphological trait measurements

    S M Rayeed, Mridul Khurana, Alyson East, Isadora E. Fluck, Elizabeth G. Campolongo, Samuel Stevens, Iuliia Zarubiieva, Scott C. Lowe, Michael W. Denslow, Evan D. Donoso, Jiaman Wu, Michelle Ramirez, Benjamin Baiser, Charles V. Stewart, Paula Mabee, Tanya Berger-Wolf, Anuj Karpatne, Hilmar Lapp, Robert P. Guralnick, Graham W. Taylor, Sydne Record
    Scientific Data (Nature Portfolio), Data Descriptor
    Project pagePaperDOIData: Hawaii-beetlesData: 2018-NEON-beetlesCode
  4. ICCV 2025 WorkshopFirst author

    Fine-Grained Beetle Taxonomy with Vision Models: A Benchmark on Long-Tailed and Domain-Adaptive Classification

    S M Rayeed, Alyson East, Samuel Stevens, Sydne Record, Charles V. Stewart
    ICCV 2025 Workshop on Computer Vision for Ecology (CV4E)
    Project pagePaperSupplement
  5. CVPR 2025 WorkshopFirst author

    BeetleVerse: A study on taxonomic classification of ground beetles

    S M Rayeed, Alyson East, Samuel Stevens, Sydne Record, Charles V. Stewart
    CVPR 2025 Workshop on CV4Animals
    PaperarXivPoster
  6. MEE 2025

    Optimizing Image Capture for Computer Vision-Powered Taxonomic Identification and Trait Recognition of Biodiversity Specimens

    Alyson East, Elizabeth G. Campolongo, Luke Meyers, S M Rayeed, Samuel Stevens, Iuliia Zarubiieva, Isadora E. Fluck, + 21 co-authors, Sydne Record
    Methods in Ecology and Evolution, 2025
    PaperarXiv

Datasets

All datasets

Hawaii-beetles

Hugging Face, DOI 10.57967/hf/7272, CC BY 4.0

Pinned carabids from NEON's Hawaii site: 162 group images, 1,614 individual specimen crops and 1,579 trait rows across 14 endemic species, with caliper and image-based elytra measurements.

Open datasetAI-ready biodiversity data

2018-NEON-beetles

Hugging Face, DOI 10.57967/hf/6890, CC BY-SA 4.0

Ethanol-preserved carabids from 30 NEON sites: 577 group images, 11,654 individual specimen images and 39,064 elytra measurement records, plus segmentation train and test splits.

Open datasetAI-ready biodiversity data

BdSL47

Mendeley Data, DOI 10.17632/pbb3w3f92y.3, CC BY 4.0

47,000 RGB images of 47 Bangla signs from 10 signers, with 470 CSV files of 21 MediaPipe hand keypoints (x, y, depth) per image.

Open datasetSign language recognition

Background

Full CV (PDF)

Education

  1. Aug 2023 to presentPhD, Computer ScienceRensselaer Polytechnic Institute
    Advisors: Charles V. Stewart and Yao Ma. CGPA 3.96/4.00
  2. Aug 2023 to May 2025MS, Computer ScienceRensselaer Polytechnic Institute
    Project: Fine-grained beetle taxonomy with vision models. CGPA 4.00/4.00
  3. Jan 2017 to Mar 2021BSc, Computer Science and EngineeringIslamic University of Technology
    CGPA 3.76/4.00

Teaching

  1. Fall 2024CSCI 4965/6965 AI for ConservationInstructor: Charles V. Stewart
  2. Fall 2023 to Fall 2024CSCI 1100 Computer Science IInstructor: Uzma Mushtaque

Experience

  1. Jan 2024 to presentGraduate Research AssistantComputer Vision Lab, RPI
    Funded by the NSF Imageomics Institute (OAC-2118240).
  2. Apr 2021 to Jul 2023Research AssociateSystems and Software Lab, IUT
    Bangla sign language recognition. Funded by UIU-IAR-01-2022-SE-37.
  3. Mar 2021 to Jul 2023Software Quality Assurance EngineerNilavo Technologies (offshore facility of Biscom Inc.)
    Test automation with Playwright, Cypress and Selenium, API and security testing.

Service

  • Reviewer, KDD AI for Science track (2026 cycle 2, 2027 cycle 1)
  • Sub-reviewer, WACV 2027 and ECCV 2026
  • Reviewer, CVPR CV4Animals workshop (2025, 2026) and NeurIPS Imageomics workshop (2025)
  • Reviewer, Communications Earth & Environment (Nature Portfolio) and PLOS ONE (2026)
  • Student organizer, Scientific Modeling Out-of-Distribution ML Challenge (2026), by Imageomics, A3D3 and iHARP

Tools I work with

VLMs
Qwen3-VL, InternVL3.5, Gemma 3/4, LLaVA-NeXT, Llama 3.2 Vision, Pixtral, Molmo2, Phi-4-multimodal, Florence-2, BLIP-2
Encoders
CLIP, SigLIP, EVA-CLIP, BioCLIP, BioCLIP-2, DINOv2, DINOv3, ViT, ConvNeXt, Swin V2
Retrieval and VQA
Multimodal RAG, dense retrieval, re-ranking, knowledge-based VQA, knowledge graph QA, multi-hop QA
Detection and segmentation
Grounding DINO, YOLOv8 to v10, SAM, SAM2, MegaDetector, Detectron2, Moondream
HPC and inference
SLURM array pipelines on A100/H100 clusters, FlashAttention, FP8 quantization, HF Transformers