kosmos

**KOSMOS** is a **multimodal large language model (MLLM) developed by Microsoft** — trained from scratch on web-scale multimodal corpora to perceive general modalities, follow instructions, and perform in-context learning (zero-shot and few-shot). **What Is KOSMOS?** - **Definition**: A "Language Is Not All You Need" foundation model. - **Architecture**: Transformer decoder (Magneto) that accepts text, audio, and image embeddings as standard tokens. - **Training**: Monolithic training on text (The Pile), image-text pairs (LAION), and interleaved data (Common Crawl). **Why KOSMOS Matters** - **raven's Matrices**: Demoed the ability to solve IQ tests (pattern completion) zero-shot. - **OCR-Free**: Reads text in images naturally without a separate OCR engine. - **Audio**: KOSMOS-1 handled vision; KOSMOS-2 and variants added grounding and speech. - **Grounding**: Can output bounding box coordinates as text tokens to localize objects. **KOSMOS** is **a true generalist model** — treating images, sounds, and text as a single unified language for the transformer to process.