dvc

**DVC** is the **data version control framework that brings Git-like reproducibility to large datasets and ML pipelines** - it tracks lightweight metadata in Git while storing heavy data artifacts in external object or file storage. **What Is DVC?** - **Definition**: Open-source tool for versioning data, models, and pipeline stages alongside code. - **Storage Pattern**: Pointers and DAG metadata stay in Git, while large files reside in S3, GCS, or local remotes. - **Pipeline Capability**: Supports reproducible stage execution with declared inputs, outputs, and dependencies. - **Workflow Outcome**: Checking out a commit can restore both code and matching data/model state. **Why DVC Matters** - **Reproducible Experiments**: Prevents hidden data drift between training runs and team members. - **Efficient Collaboration**: Developers share data lineage without committing large binaries to Git. - **Pipeline Reliability**: Dependency graph tracking makes rebuilds explicit and deterministic. - **Cost Control**: Remote cache reuse avoids repeated full data copies across environments. - **MLOps Readiness**: Provides practical bridge between notebook experimentation and production pipelines. **How It Is Used in Practice** - **Repo Initialization**: Track datasets and model artifacts with DVC metadata files committed to Git. - **Remote Configuration**: Configure secure shared storage backend for artifact push and pull operations. - **Pipeline Governance**: Define dvc.yaml stages and integrate checks into CI before model promotion. DVC is **a practical foundation for reproducible data-centric ML development** - it extends source-control discipline to the large artifacts that actually drive model behavior.