dataset versioning

**Dataset versioning** is the **practice of creating immutable, traceable dataset snapshots for every training and evaluation run** - it ensures model results can be reproduced even when underlying raw data continues to evolve. **What Is Dataset versioning?** - **Definition**: Controlled lifecycle management of dataset states with unique identifiers and metadata. - **Version Scope**: Includes raw data, preprocessing outputs, label revisions, and split definitions. - **Lineage Model**: Links each dataset version to source systems, transformation code, and quality checks. - **Operational Output**: A run can always resolve the exact data state used for training or validation. **Why Dataset versioning Matters** - **Reproducibility**: Without fixed data versions, retraining can silently produce different model behavior. - **Auditability**: Version history supports compliance, governance, and incident root-cause analysis. - **Experiment Integrity**: Model comparisons are meaningful only when dataset differences are explicit. - **Rollback Safety**: Teams can revert quickly to prior trusted data states when quality regressions appear. - **Collaboration**: Shared immutable references prevent confusion across research and platform teams. **How It Is Used in Practice** - **Snapshot Policy**: Create immutable dataset versions at major ingestion, labeling, and preprocessing milestones. - **Metadata Capture**: Store schema, statistics, data-source hashes, and transformation commit IDs per version. - **Run Binding**: Require every experiment log and model artifact to reference a concrete dataset version ID. Dataset versioning is **a core control for reliable ML lifecycle management** - immutable data references are essential for reproducible science and trustworthy deployment decisions.