bioasq
**BioASQ** is the **large-scale biomedical question answering and information retrieval challenge** — running since 2013 as an annual shared task requiring systems to retrieve relevant PubMed articles, extract exact answer snippets, and generate well-formed natural language answers to biomedical research questions, directly targeting the information overload problem in scientific literature.
**What Is BioASQ?**
- **Origin**: Tsatsaronis et al. (2015); annual challenge run by the BioASQ organization.
- **Scale**: 4,234+ biomedical questions (growing annually); linked to the full PubMed corpus (35M+ articles).
- **Format**: Expert-formulated questions by biomedical scientists + gold standard annotations for relevant documents, snippets, exact answers, and ideal answers.
- **Question Types**: Yes/No, Factoid (single entity answer), List (multiple entities), Summary (paragraph answer).
- **Challenge Phases**: Phase A (document and snippet retrieval) and Phase B (answer generation).
**The Four Question Types**
**Yes/No**: "Is the protein BRCA1 involved in DNA repair?" → "yes" + supporting snippets.
**Factoid**: "What is the mechanism of action of imatinib?" → "selective BCR-ABL tyrosine kinase inhibitor" + exact snippet spans.
**List**: "Which genes are known to be associated with cystic fibrosis?" → ["CFTR", "TGFB1", "MUC5B", ...] + supporting documents.
**Summary**: "What is known about the role of PCSK9 in cholesterol metabolism?" → Multi-sentence synthesized answer from retrieved literature.
**Why BioASQ Is Hard**
- **Biomedical Terminology**: Questions use precise MESH/UMLS terminology ("phospholipase A2 group VII" not "platelet-activating factor acetylhydrolase"). Systems must handle synonym explosion in biomedical nomenclature.
- **Literature Scale**: PubMed grows by ~1 million articles per year. Systems must retrieve the relevant needle from 35M+ papers.
- **Multi-Hop Evidence**: Summary questions require synthesizing findings from multiple conflicting or complementary studies.
- **Answer Granularity**: For factoid questions, the exact answer span (gene name, drug name, measurement value) must be extracted — not just the document.
- **Scientific Precision**: "Which kinase phosphorylates Ser473 of AKT?" has a specific correct answer (PDK2/mTORC2) with no tolerance for close-but-wrong responses.
**Performance Results (BioASQ Phase B)**
| System | Factoid MRR | List F1 | Yes/No Accuracy | Summary ROUGE |
|--------|------------|---------|-----------------|---------------|
| IR baseline | 0.22 | 0.31 | 72% | 0.28 |
| BioBERT fine-tuned | 0.48 | 0.49 | 81% | 0.38 |
| PubMedBERT | 0.51 | 0.52 | 83% | 0.41 |
| GPT-4 + RAG (PubMed) | 0.62 | 0.58 | 87% | 0.52 |
| BioGPT (domain-pretrained) | 0.66 | 0.60 | 88% | 0.55 |
**Why BioASQ Matters**
- **Research Acceleration**: Scientists spend ~20% of their work time searching literature. BioASQ-capable systems can instantly synthesize the current evidence base for any biomedical question.
- **Clinical Evidence Retrieval**: At the point of care, physicians need rapid answers to specific drug-mechanism, dosing, or interaction questions — BioASQ tests exactly this capability.
- **Drug Discovery Applications**: "Which proteins interact with target X?" and "Which compounds inhibit pathway Y?" are BioASQ-style queries for computational drug target identification.
- **Systematic Review Foundation**: Literature-grounded QA systems can semi-automate the retrieval and evidence extraction phases of systematic reviews.
- **Domain Pretraining Validation**: BioASQ is the primary benchmark validating that BioBERT, PubMedBERT, BioGPT, and BioMedLM outperform generic models — demonstrating the value of biomedical corpus pretraining.
BioASQ is **the biomedical literature intelligence test** — measuring whether AI can navigate the 35 million papers of PubMed to retrieve, extract, and synthesize precise scientific answers to the questions that drive biomedical research and clinical evidence-based practice.