Vision models identify 1,769 ground beetle species at 94% accuracy from curated images, but lose up to 37% in the field. We built the benchmark, the dataset and the pipelines behind that finding.
Biodiversity monitoring depends on identifying and measuring organisms at scale, a task where classes differ in fine detail, labels follow a long tail and field images look nothing like lab images. Ground beetles, sensitive indicators of ecosystem change monitored across the United States by NEON, have all of these properties.
This line of work covers the full path: imaging specimens and validating their measurements, extracting each specimen from tray photographs with open-vocabulary detection and segmentation, and benchmarking which vision encoders can recognize them.
12 self-supervised, supervised and vision-language encoders, linear-probed on 100,885 images of 1,769 species from four long-tailed datasets. ViLT reaches 97.15% genus and 93.97% species accuracy; half the training data costs little accuracy; transfer from lab to field images drops by up to 37%.
13,200+ NEON ground beetles from 30 sites, imaged at high resolution, extracted from group images with Grounding DINO and measured: 39,064 elytra records whose image measurements agree with calipers at R² 0.94 to 0.98.
Three stages for tray images of pinned beetles: detect every specimen, crop and sort it with its metadata, and segment it into body parts. Detection matches the exact count on 97.81% of 1,506 NEON trays.
With hundreds to thousands of labels, the regime most science lives in, multimodal LLMs plateau early while vision-only encoders keep improving. On NeWT, 10x more training samples beats 10x more inference compute.
Pinned carabids from NEON's Hawaii site: 162 group images, 1,614 individual specimen crops and 1,579 trait rows across 14 endemic species, with caliper and image-based elytra measurements.
Ethanol-preserved carabids from 30 NEON sites: 577 group images, 11,654 individual specimen images and 39,064 elytra measurement records, plus segmentation train and test splits.
