galactica
**Galactica** is a **120 billion parameter open-source language model trained on scientific and academic texts from arXiv, PubMed, mathematics repositories, and academic papers by Meta AI**, designed specifically for scientific reasoning and knowledge retrieval—pioneering domain-specialized frontier-scale LLMs and exploring whether models trained on pure high-quality academic data outperform general internet-trained models on intellectual tasks.
**Scientific Text Specialization**
| Training Data | Quantity | Purpose |
|--------------|----------|---------|
| arXiv papers | Scientific preprints | Physics, ML, mathematics |
| PubMed | Biomedical literature | Medicine, biology research |
| Mathematics | Symbolic reasoning | Equation understanding |
| Academic papers | Peer-reviewed knowledge | Quality-filtered information |
Galactica was trained **exclusively** on high-quality-curated scientific and academic sources—a radical departure from web-scale models trained on noisy internet data.
**Novel Capabilities**: Galactica introduced **scientific prompting**:
- Citation generation (predicting relevant academic references)
- Equation understanding (recognizing when models misunderstand math)
- Table and figure interpretation from papers
**Intended Purpose**: Enable scientists to query scientific literature as natural language, enabling retrieval and reasoning across millions of papers—essentially making AI assistants for scientific research.
**Reception & Lessons**: Galactica was controversially released then quickly withdrawn when researchers documented concerning errors (hallucinating fake papers and citations). This taught the community valuable lessons about risk assessment for specialized models.
**Legacy**: Despite challenges, Galactica inspired the domain-specialized LLM trend. Models like Falcon (code-optimized, Stable Diffusion for text, etc.) followed the principle that **training on curated domain data produces better specialist models**.