orca mini
**Orca Mini** is a **series of small language models (3B, 7B) applying Microsoft's Orca methodology (explanation-based training) to smaller base models, proving that reasoning capabilities can be learned by students models at any scale** — demonstrating that instruction-tuning with detailed step-by-step reasoning traces enables even tiny models to achieve surprising logical competence and teaching ability beyond their raw parameter count.
**The Orca Methodology Scaled Down**
Orca Mini adapts the full Orca approach to resource-constrained settings:
- **Explanation Tuning**: Train on reasoning traces showing step-by-step logic, not just final answers
- **Student Model Learning**: Capture teacher reasoning patterns in compressed form
- **On-Device Reasoning**: Enable logical inference on phones/laptops with <10B parameters
| Model Version | Parameters | Use Case | Advantage |
|--------------|-----------|----------|-----------|
| **Orca Mini 3B** | 3 billion | Mobile devices, edge | Fits on-device, reasoning capable |
| **Orca Mini 7B** | 7 billion | Laptops/servers | Better reasoning quality than larger models |
**Impact**: Proved that **reasoning ability transcends scale**—a 3B Orca Mini with explanation training outperforms much larger models trained on raw datasets. This influenced the entire small language model movement.