supervised
**Supervised Fine-Tuning (SFT)**
**What is SFT?**
Supervised Fine-Tuning trains a pretrained LLM on curated (instruction, response) pairs to follow instructions and produce helpful outputs. It is typically the first step after pretraining.
**Data Format**
```json
{
"instruction": "Write a haiku about programming",
"input": "",
"output": "Lines of code flow down
Debugging through the night hours
Compiler agrees"
}
```
Or in conversation format:
```json
{
"messages": [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Write a haiku about programming"},
{"role": "assistant", "content": "Lines of code flow down
Debugging through the night hours
Compiler agrees"}
]
}
```
**Dataset Recommendations**
**Dataset Sizes**
| Use Case | Recommended Size |
|----------|------------------|
| Domain adaptation | 1K-10K examples |
| Instruction following | 10K-50K examples |
| Full capability tuning | 50K-500K examples |
**Popular Open Datasets**
| Dataset | Size | Focus |
|---------|------|-------|
| OpenAssistant/oasst1 | 161K | Multi-turn conversations |
| Dolly-15K | 15K | Diverse instructions |
| Alpaca-52K | 52K | GPT-generated instructions |
| WizardLM | 196K | Complex instruction evolution |
| CodeAlpaca | 20K | Coding tasks |
**SFT Best Practices**
1. **Quality over quantity**: 1K excellent examples > 100K mediocre ones
2. **Diversity**: Cover wide range of tasks and formats
3. **Formatting consistency**: Same structure across examples
4. **Response length**: Match desired output length distribution
5. **Human review**: Verify a sample of training data manually
**Training Considerations**
- Epochs: 1-3 (avoid overfitting)
- Learning rate: 1e-5 to 5e-5 for full fine-tuning
- Use LoRA/QLoRA for parameter-efficient training
- Validate on held-out set to monitor overfitting