synthetic data generation

Synthetic data generation creates artificial training data using generative models to augment or replace real data. **Motivations**: Insufficient real data, privacy constraints, rare event simulation, cost of data collection, label automation. **Approaches**: **LLM generation**: Prompt models to generate text examples, dialogues, QA pairs. **Diffusion models**: Generate images with controlled attributes. **GANs**: Generate realistic samples from learned distribution. **Simulation**: Physics/rule-based synthetic environments. **For NLP**: Generate paraphrases, translations, domain-specific text, instruction-response pairs, code examples. **Quality considerations**: Diversity (avoid mode collapse), fidelity (realistic enough to be useful), correct labels, avoiding generating training data artifacts. **Self-instruct pattern**: LLM generates instruction-following examples for its own training. **Bootstrapping**: Generate data, train model, use improved model to generate better data. **Challenges**: Distribution shift from real data, error amplification, potential for generating harmful content. **Tools**: OpenAI API, Alpaca-style generation, Evol-Instruct. Increasingly central to LLM training pipelines.