Galactica is a 120 billion parameter open-source language model trained on scientific and academic texts from arXiv, PubMed, mathematics repositories, and academic papers by Meta AI, designed specifically for scientific reasoning and knowledge retrieval—pioneering domain-specialized frontier-scale LLMs and exploring whether models trained on pure high-quality academic data outperform general internet-trained models on intellectual tasks.

Scientific Text Specialization

Galactica was trained exclusively on high-quality-curated scientific and academic sources—a radical departure from web-scale models trained on noisy internet data.

Novel Capabilities: Galactica introduced scientific prompting:

• Citation generation (predicting relevant academic references)
• Equation understanding (recognizing when models misunderstand math)
• Table and figure interpretation from papers

Intended Purpose: Enable scientists to query scientific literature as natural language, enabling retrieval and reasoning across millions of papers—essentially making AI assistants for scientific research.

Reception & Lessons: Galactica was controversially released then quickly withdrawn when researchers documented concerning errors (hallucinating fake papers and citations). This taught the community valuable lessons about risk assessment for specialized models.

Legacy: Despite challenges, Galactica inspired the domain-specialized LLM trend. Models like Falcon (code-optimized, Stable Diffusion for text, etc.) followed the principle that training on curated domain data produces better specialist models.