dolly

**Dolly (Databricks)** **Overview** Dolly (specifically Dolly 2.0) was a pivotal open-source Large Language Model released by Databricks in April 2023. It was the first LLM to feature a completely open-source, commercially usable instruction-tuning dataset. **The Innovation: databricks-dolly-15k** Before Dolly, most open models (like Alpaca) were trained on data generated by ChatGPT. The OpenAI Terms of Service forbade using this output to train competing commercial models. Databricks created a dataset of 15,000 high-quality prompts/responses written *by their own employees* (clean, human-generated data). - **License**: Creative Commons (CC-BY-SA). Anyone can use it for commercial purposes. **The Model** - **Base**: Pythia-12B. - **Training**: Fine-tuned on the 15k dataset. - **Capabilities**: Brainstorming, classification, generation. **Legacy** Dolly was not the smartest model, but it proved that **Data Quality > Data Quantity**. A small, high-quality dataset could give a model "instruction following" capabilities better than massive noisy datasets. It kicked off the "Open Source Commercial LLM" wave.