Supervised Fine-Tuning (SFT)

What is SFT? Supervised Fine-Tuning trains a pretrained LLM on curated (instruction, response) pairs to follow instructions and produce helpful outputs. It is typically the first step after pretraining.

Data Format

{
  "instruction": "Write a haiku about programming",
  "input": "",
  "output": "Lines of code flow down
Debugging through the night hours
Compiler agrees"
}

Or in conversation format:

{
  "messages": [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Write a haiku about programming"},
    {"role": "assistant", "content": "Lines of code flow down
Debugging through the night hours
Compiler agrees"}
  ]
}

Dataset Recommendations

Dataset Sizes

Popular Open Datasets

SFT Best Practices 1. Quality over quantity: 1K excellent examples > 100K mediocre ones 2. Diversity: Cover wide range of tasks and formats 3. Formatting consistency: Same structure across examples 4. Response length: Match desired output length distribution 5. Human review: Verify a sample of training data manually

Training Considerations

• Epochs: 1-3 (avoid overfitting)
• Learning rate: 1e-5 to 5e-5 for full fine-tuning
• Use LoRA/QLoRA for parameter-efficient training
• Validate on held-out set to monitor overfitting