dagger

**DAgger** (Dataset Aggregation) is an **imitation learning algorithm that addresses behavioral cloning's distribution shift problem** — iteratively collecting new expert labels for the states the LEARNER visits, aggregating them into the training dataset, and retraining the policy. **DAgger Algorithm** - **Step 1**: Train initial policy $pi_1$ via behavioral cloning on expert demonstrations $D$. - **Step 2**: Roll out $pi_i$ to collect states visited by the learner. - **Step 3**: Query the expert for the correct actions at these states — get expert labels for learner-visited states. - **Step 4**: Aggregate: $D leftarrow D cup D_{new}$, retrain $pi_{i+1}$. Repeat. **Why It Matters** - **Distribution Shift Fix**: By training on states the LEARNER visits (not just expert states), DAgger eliminates distribution shift. - **Theoretical**: DAgger provides no-regret guarantees — the learned policy converges to expert performance. - **Interactive**: Requires an interactive expert who can label learner states — not always available. **DAgger** is **learning from your own mistakes** — iteratively getting expert feedback on the states the learner actually visits.