decision tree extraction

**Decision Tree Extraction** is a **model distillation technique that trains a decision tree to approximate the predictions of a complex model** — producing an interpretable tree-structured model that captures the essential decision logic of the original neural network or ensemble. **Extraction Methods** - **Soft Labels**: Train a decision tree using the complex model's predicted probabilities as soft targets. - **Born-Again Trees**: Iteratively refine the tree using the complex model's outputs on synthetic data. - **Neural-Backed Trees**: Embed neural network features into tree decision nodes for richer splits. - **Pruning**: Aggressively prune to keep the tree small enough for human interpretation. **Why It Matters** - **Interpretability**: Decision trees are among the most interpretable model types — clear decision paths. - **Fidelity vs. Complexity**: Balance between faithfully approximating the complex model and keeping the tree small. - **Regulatory**: Some industries require model explanations in tree/rule form for compliance. **Decision Tree Extraction** is **simplifying complexity into a tree** — distilling a complex model's decisions into an interpretable tree structure.