AAAI 2027 WorkshopUnder reviewFirst author

LABEL: Literature Asymmetry between Efficacy and Limits in Drug Knowledge and Language Models

S M Rayeed, Charles V. Stewart, Yao Ma

arXiv preprint, 2026. Under review at the AAAI 2027 Workshop on Health Intelligence (W3PHIAI)

LABEL measurement design: PrimeKG drug-disease edges, an evidence search over Wikipedia, PubMed and the FDA label, seven language models tested closed book and with the drug's page, and within-drug comparisons of indications and contraindications.
LABEL measurement design: PrimeKG drug-disease edges, an evidence search over Wikipedia, PubMed and the FDA label, seven language models tested closed book and with the drug's page, and within-drug comparisons of indications and contraindications.

Prose names what a drug treats about twice as often as who must not take it, and language models inherit the gap. The FDA label shows no gap, so routing safety questions to the label recovers the missing facts.

Every drug has two kinds of fact: its efficacy, what it treats, and its limits, who must not take it. Naproxen's Wikipedia page names gout and rheumatoid arthritis but not active peptic ulcer disease. A wrong limit can hurt a patient, yet people write about benefits far more than risks, and large language models learn from that writing.

We measure how often Wikipedia, PubMed and the FDA label name each drug fact, comparing indications and contraindications inside the same drug so a famous drug cannot skew the result. We then test seven language models on the same facts, closed book and with a retrieved passage, and test a routing rule that sends safety questions to the label.

Study at a glance

Facts574 PrimeKG drugs with 3 indications and 3 contraindications each; disease families from MONDO
SourcesWikipedia, PubMed abstracts and the whole FDA label (472 drugs)
MatchingExact name, alternative name, family only or absent, with a random floor for chance matches and an LLM judge checked against four annotators (Fleiss' κ 0.89 and 0.92)
ModelsQwen2.5 from 0.5B to 14B, OLMo-2 7B and Granite-3.1 8B
QuestionsFive-option closed-book questions, plus 856 harder questions with sibling distractors and “Not enough information”
ControlHetionet treats against palliates, two positive relations
Text names a drug's indications about twice as often as its contraindications in Wikipedia and PubMed.
Text names a drug's indications about twice as often as its contraindications in Wikipedia and PubMed.

Findings

  1. Prose names uses about twice as often as limits. The within-drug odds ratio is 2.43 in Wikipedia, 2.77 in PubMed and 2.19 for both together.
  2. The FDA label has no gap. The whole label names uses and limits at the same rate (odds ratio 1.05, p = 0.62), so the gap comes from the prose, not the matcher. Most limits sit in the Boxed Warning and Warnings sections, not the Contraindications section.
  3. Bigger models learn uses, not limits. From Qwen2.5 0.5B to 14B, accuracy on indications rises by 18.3 points; accuracy on contraindications stays near the 20% chance level.
  4. Models read safety facts well when they are given. A passage that names the condition helps limits at least as much as uses (+14.2 against +11.1 points). Family-only and silent passages do not help.
  5. The gap is specific to risk. Two positive relations in Hetionet, treats and palliates, show no gap (odds ratio 1.05).
  6. Routing to the label helps when the label names the fact. On those safety questions, a label passage raises accuracy from 40.0% to 55.8%.
Scaling Qwen2.5 from 0.5B to 14B raises indication accuracy by 18.3 points; contraindication accuracy stays near the 20% chance level.
Scaling Qwen2.5 from 0.5B to 14B raises indication accuracy by 18.3 points; contraindication accuracy stays near the 20% chance level.
A passage helps only when it names the condition; family-only and silent passages do not help.
A passage helps only when it names the condition; family-only and silent passages do not help.
The whole FDA label shows no gap (odds ratio 1.05), although its Contraindications section alone names few of these limits.
The whole FDA label shows no gap (odds ratio 1.05), although its Contraindications section alone names few of these limits.
With sibling distractors and a decline option, models fail on contraindications in two styles: some guess, others abstain.
With sibling distractors and a decline option, models fail on contraindications in two styles: some guess, others abstain.
A relation-aware agent: a planner routes safety questions to regulatory records, retrievers query the FDA label and DrugCentral, and a verifier answers with a source, expands and retries, or declines.
A relation-aware agent: a planner routes safety questions to regulatory records, retrievers query the FDA label and DrugCentral, and a verifier answers with a source, expands and retries, or declines.

Related work

  1. AAAI 2027 WorkshopUnder reviewFirst author

    CLADE: Category-Level Answering from Deficient Evidence in Retrieval-Augmented Language Models

    S M Rayeed, Charles V. Stewart, Yao Ma
    arXiv preprint, 2026. Under review at the AAAI 2027 NeuSymBridge Workshop
  2. ECCV 2026

    Identifying and Resolving Pitfalls of Knowledge-Based VQA Benchmarks: Auditing, Repairing, and Augmenting

    Qian Ma, S M Rayeed, Charles V. Stewart, Qiong Wu, Yao Ma
    European Conference on Computer Vision (ECCV 2026)
  3. WACV 2027First author

    How Finely Can Vision-Language Models See Biology? Bio-VQA: A Benchmark for Taxonomic, Morphological, Spatial and Individual Reasoning

    S M Rayeed, Qian Ma, Charles V. Stewart, Yao Ma
    IEEE/CVF Winter Conference on Applications of Computer Vision (WACV 2027)

Cite

@article{rayeed2026label,
  title   = {LABEL: Literature Asymmetry between Efficacy and Limits in Drug Knowledge and Language Models},
  author  = {Rayeed, S M and Stewart, Charles V. and Ma, Yao},
  journal = {arXiv preprint arXiv:2611.19166},
  year    = {2026}
}