llama

LLaMA (Large Language Model Meta AI) is Metas open-source foundation model family that democratized LLM research. **Significance**: First truly capable open-weights LLM, enabled explosion of open-source AI research and applications. **LLaMA 1 (Feb 2023)**: 7B, 13B, 33B, 65B parameters. Trained on public data only. Matched GPT-3 quality at smaller sizes. **Architecture**: Standard decoder-only transformer with pre-normalization (RMSNorm), SwiGLU activation, rotary embeddings (RoPE), no bias terms. **Training data**: 1.4T tokens from CommonCrawl, C4, GitHub, Wikipedia, Books, ArXiv, StackExchange. **Efficiency focus**: Designed for inference efficiency, smaller models matching larger ones through better data and training. **Open ecosystem**: Spawned Alpaca, Vicuna, and hundreds of fine-tuned variants. **Research impact**: Enabled academic research on LLM behavior, fine-tuning, alignment. **Limitations**: Original release research-only license, limited commercial use. **Legacy**: Changed the landscape of open AI, proved open models could compete with proprietary ones.